Color index coding for palette-based video coding

ABSTRACT

In palette-based coding, a video coder may form a so-called “palette” as a table of colors representing the video data of a given block. The video coder may code index values for one or more pixels values of a current block of video data, where the index values indicate entries in the palette that represent the pixel values of the current block. A method includes determining a palette for a block of video data, identifying escape pixel(s) not associated with any palette entries, identifying a single quantization parameter (QP) value for all escape pixels of the block for a given color channel using a QP value for non-palette based coding of transform coefficients, dequantizing each escape pixel using the identified QP value, and determining pixel values of the block using the dequantized escape pixels and index values for any pixel(s) associated with any palette entries.

This application claims the benefit of:

U.S. Provisional Patent Application No. 61/923,163, filed 2 Jan. 2014;

U.S. Provisional Patent Application No. 61/924,141, filed 6 Jan. 2014;and

U.S. Provisional Application No. 61/969,759, filed 24 Mar. 2014, theentire contents of each of which are incorporated herein by reference.

TECHNICAL FIELD

This disclosure relates to video encoding and decoding.

BACKGROUND

Digital video capabilities can be incorporated into a wide range ofdevices, including digital televisions, digital direct broadcastsystems, wireless broadcast systems, personal digital assistants (PDAs),laptop or desktop computers, tablet computers, e-book readers, digitalcameras, digital recording devices, digital media players, video gamingdevices, video game consoles, cellular or satellite radio telephones,so-called “smart phones,” video teleconferencing devices, videostreaming devices, and the like. Digital video devices implement videocompression techniques, such as those described in the standards definedby MPEG-2, MPEG-4, ITU-T H.263, ITU-T H.264/MPEG-4, Part 10, AdvancedVideo Coding (AVC), the High Efficiency Video Coding (HEVC) standardpresently under development, and extensions of such standards. The videodevices may transmit, receive, encode, decode, and/or store digitalvideo information more efficiently by implementing such videocompression techniques.

Video compression techniques perform spatial (intra-picture) predictionand/or temporal (inter-picture) prediction to reduce or removeredundancy inherent in video sequences. For block-based video coding, avideo slice (i.e., a video frame or a portion of a video frame) may bepartitioned into video blocks. Video blocks in an intra-coded (I) sliceof a picture are encoded using spatial prediction with respect toreference samples in neighboring blocks in the same picture. Videoblocks in an inter-coded (P or B) slice of a picture may use spatialprediction with respect to reference samples in neighboring blocks inthe same picture or temporal prediction with respect to referencesamples in other reference pictures. Pictures may be referred to asframes, and reference pictures may be referred to as reference frames.

Spatial or temporal prediction results in a predictive block for a blockto be coded. Residual data represents pixel differences between theoriginal block to be coded and the predictive block. An inter-codedblock is encoded according to a motion vector that points to a block ofreference samples forming the predictive block, and the residual dataindicates the difference between the coded block and the predictiveblock. An intra-coded block is encoded according to an intra-coding modeand the residual data. For further compression, the residual data may betransformed from the pixel domain to a transform domain, resulting inresidual coefficients, which then may be quantized. The quantizedcoefficients, initially arranged in a two-dimensional array, may bescanned in order to produce a one-dimensional vector of coefficients,and entropy coding may be applied to achieve even more compression.

A multiview coding bitstream may be generated by encoding views, e.g.,from multiple perspectives. Some three-dimensional (3D) video standardshave been developed that make use of multiview coding aspects. Forexample, different views may transmit left and right eye views tosupport 3D video. Alternatively, some 3D video coding processes mayapply so-called multiview plus depth coding. In multiview plus depthcoding, a 3D video bitstream may contain not only texture viewcomponents, but also depth view components. For example, each view maycomprise one texture view component and one depth view component.

SUMMARY

In general, techniques of this disclosure relate to palette-based videocoding. In palette-based coding, a video coder (e.g., a video encoder ora video decoder) may form a so-called “palette” as a table of colors orpixel values representing the video data of a particular area (e.g., agiven block). In this way, rather than coding actual pixel values ortheir residuals for a current block of video data, the video coder maycode color or palette index values for one or more of the pixels valuesof the current block, where the index values indicate entries in thepalette that are used to represent the pixel values of the currentblock. A map of palette index values for a current block of video datamay be coded line by line using a given scan order and run-length codingtechniques. Each of the index values in a given line of the map may beexplicitly coded, predicted from a left-mode index of the given line, orpredicted from a collocated index in a line above the given line.

Various techniques of this disclosure are directed to enhancing existingpalette-based coding techniques. In some aspects, this disclosure isdirected to techniques for bypassing coding of a map of palette indexvalues for a block, if the block meets certain criteria. In someaspects, this disclosure is directed to determining a maximum range ofvalues (also referred to herein as an “error limit”) for a given paletteusing a mapping table that stores a relationship between quantizationparameter values and palette error limits. In some aspects, thisdisclosure is directed to defining a quantization parameter for pixelsof a palette-coded block that do not map to an entry in thecorresponding palette (referred to herein as “escape pixels”) based onquantization parameters used for traditional coefficient coding in acorresponding color channel.

In one example, this disclosure is directed to a method of decodingvideo data, the method including determining a number of entriesincluded in a palette used to represent pixel values of a block of videodata to be decoded, and determining whether the block of video dataincludes at least one escape pixel that is not associated with any ofthe entries in the palette. The method may further include responsive todetermining that the number of entries included in the palette is equalto one and that the block of video data does not include at least oneescape pixel, bypassing decoding of index values associated with thepalette for the pixel values of the block of video data, and determiningthe pixel values of the block of video data to be equal to the one entryincluded in the palette.

In another example, this disclosure is directed to a method of encodingvideo data, the method including determining a number of entriesincluded in a palette used to represent pixel values of a block of videodata to be encoded, and determining whether the block of video dataincludes at least one escape pixel that is not associated with any ofthe entries in the palette. The method may further include responsive todetermining that the number of entries included in the palette is equalto one and that the block of video data does not include at least oneescape pixel, bypassing encoding of index values associated with thepalette for the pixel values of the block of video data, and encodingthe block of video data by determining the pixel values of the block ofvideo data to be equal to the one entry included in the palette.

In another example, this disclosure is directed to an apparatus fordecoding video data, the apparatus comprising a memory configured tostore video data, and one or more processors configured to determine anumber of entries included in a palette used to represent pixel valuesof a block of video data to be coded, and to determine whether the blockof video data includes at least one escape pixel that is not associatedwith any of the entries in the palette. The one or more processors mayfurther be configured to responsive to a determination that the numberof entries included in the palette is equal to one and that the block ofvideo data does not include at least one escape pixel, bypass coding ofindex values associated with the palette for the pixel values of theblock of video data, and to determine the pixel values of the block ofvideo data to be equal to the one entry included in the palette.

In another example, this disclosure is directed toward an apparatus forcoding video data, the apparatus comprising means for determining anumber of entries included in a palette used to represent pixel valuesof a block of video data to be coded, and means for determining whetherthe block of video data includes at least one escape pixel that is notassociated with any of the entries in the palette. The apparatus mayfurther include means for bypassing, responsive to determining that thenumber of entries included in the palette is equal to one and that theblock of video data does not include at least one escape pixel, codingof index values associated with the palette for the pixel values of theblock of video data and means for determining the pixel values of theblock of video data to be equal to the one entry included in thepalette.

In another example, this disclosure is directed toward a non-transitorycomputer-readable medium encoded with instructions that that, whenexecuted, cause one or more processors a device for coding video data todetermine a number of entries included in a palette used to representpixel values of a block of video data to be coded, and to determinewhether the block of video data includes at least one escape pixel thatis not associated with any of the entries in the palette. Theinstructions, when executed, may further cause the one or moreprocessors to responsive to a determination that the number of entriesincluded in the palette is equal to one and that the block of video datadoes not include at least one escape pixel, bypass coding of indexvalues associated with the palette for the pixel values of the block ofvideo data, and to determine the pixel values of the block of video datato be equal to the one entry included in the palette.

In one example, this disclosure is directed to a method of decodingvideo data, the method including determining a palette used to representpixel values of a block of video data to be decoded, and identifying, inthe block of video data, one or more escape pixels that are notassociated with any of one or more entries in the palette. The methodmay further include identifying a single quantization parameter (QP)value for all of the one or more escape pixels of the block for a givencolor channel based on a QP value used for transform coefficient codingin non-palette based coding, and dequantizing each of the one or moreescape pixels using the identified single QP value. The method mayfurther include determining the pixel values of the block of video databased on the dequantized escape pixels and index values received for oneor more pixels in the block of video data that are associated with theone or more entries in the palette.

In another example, this disclosure is directed to a method of encodingvideo data, the method including determining a palette used to representpixel values of a block of video data to be encoded, and identifying, inthe block of video data, one or more escape pixels that are notassociated with any of the one or more entries in the palette. Themethod may further include identifying a single quantization parameter(QP) value for all of the one or more escape pixels of the block for agiven color channel based on a QP value used for transform coefficientcoding in non-palette based coding, and quantizing each of the one ormore escape pixels using the identified single QP value. The method mayfurther include encoding the pixel values of the block of video dataincluding the quantized escape pixels and index values for one or morepixels in the block of video data that are associated with the one ormore entries in the palette.

In another example, this disclosure is directed to an apparatus forcoding video data, the apparatus comprising a memory configured to storevideo data, and one or more processors in communication with the memoryand configured to determine a palette used to represent pixel values ofa block of video data to be coded, and to identify, in the block ofvideo data, one or more escape pixels that are not associated with anyof one or more entries in the palette. The one or more processors may befurther configured to identify a single quantization parameter (QP)value for all of the one or more escape pixels of the block for a givencolor channel based on a QP value used for transform coefficient codingin non-palette based coding, and to apply the identified single QP valueto each of the one or more escape pixels. The one or more processors maybe further configured to determine the pixel values of the block ofvideo data based on the escape pixels and index values received for oneor more pixels in the block of video data that are associated with theone or more entries.

In another example, this disclosure is directed toward an apparatus forcoding video data, the apparatus comprising means for means fordetermining a palette used to represent pixel values of a block of videodata to be coded, means for identifying, in the block of video data, oneor more escape pixels that are not associated with any of one or moreentries in the palette, means for identifying a single quantizationparameter (QP) value for all of the one or more escape pixels of theblock for a given color channel based on a QP value used for transformcoefficient coding in non-palette based coding, means for applying theidentified single QP value to each of the one or more escape pixels, andmeans for determining the pixel values of the block of video data basedon the escape pixels and index values received for one or more pixels inthe block of video data that are associated with the one or moreentries.

In another example, this disclosure is directed toward a non-transitorycomputer-readable medium encoded with instructions that that, whenexecuted, cause one or more processors a device for coding video data todetermine a palette used to represent pixel values of a block of videodata to be coded, to identify, in the block of video data, one or moreescape pixels that are not associated with any of one or more entries inthe palette, and to identify a single quantization parameter (QP) valuefor all of the one or more escape pixels of the block for a given colorchannel based on a QP value used for transform coefficient coding innon-palette based coding. The instructions, when executed, may furthercause the one or more processors to apply the identified single QP valueto each of the one or more escape pixels, and to determine the pixelvalues of the block of video data based on the escape pixels and indexvalues received for one or more pixels in the block of video data thatare associated with the one or more entries.

The techniques described herein may provide one or more potentialadvantages and improvements over existing palette-based codingtechniques and/or data compression techniques. For instance, varioustechniques of this disclosure may be implemented by video coding devicesto conserve computing resources and bandwidth requirements, whilemaintaining data precision. Additionally, various techniques of thisdisclosure may be implemented by video coding devices to improve theefficiency and accuracy of existing palette-based coding techniques anddata compression techniques.

The details of one or more examples of the disclosure are set forth inthe accompanying drawings and the description below. Other features,objects, and advantages will be apparent from the description, drawings,and claims.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram illustrating an example video coding systemthat may utilize the techniques described in this disclosure.

FIG. 2 is a block diagram illustrating an example video encoder that mayimplement the techniques described in this disclosure.

FIG. 3 is a block diagram illustrating an example video decoder that mayimplement the techniques described in this disclosure.

FIG. 4 is a flowchart illustrating an example process by which a videodecoding device may implement techniques of this disclosure to bypassdecoding of index values for pixels of a palette-coded block, based on aparticular set of conditions.

FIG. 5 is a flowchart an example process by which a video encodingdevice may implement techniques of this disclosure to bypass encoding ofindex values for pixels of a palette-coded block, based on a particularset of conditions.

FIG. 6 is a flowchart illustrating an example process by which a videodecoding device may implement techniques of this disclosure todequantize one or more escape pixels of a palette-coded block of videodata.

FIG. 7 is a flowchart illustrating an example process by which a videoencoding device may implement techniques of this disclosure to quantizeone or more escape pixels of a palette-coded block of video data.

DETAILED DESCRIPTION

This disclosure includes techniques for video coding and compression. Inparticular, this disclosure describes techniques for palette-basedcoding of video data. In traditional video coding, images are assumed tobe continuous-tone and spatially smooth. Based on these assumptions,various tools have been developed, such as block-based transform,filtering, etc., and such tools have shown good performance for naturalcontent videos.

In applications like remote desktop, collaborative work and wirelessdisplay, however, computer generated screen content (e.g., such as textor computer graphics) may be the dominant content to be compressed. Thistype of content tends to have discrete-tone, and feature sharp lines andhigh-contrast object boundaries. The assumption of continuous-tone andsmoothness may no longer apply for screen content, and thus traditionalvideo coding techniques may not be efficient ways to compress video dataincluding screen content.

This disclosure describes palette-based coding, which may beparticularly suitable for screen generated content coding. For example,assuming that a particular area of video data has a relatively smallnumber of colors, a video coder (e.g., a video encoder or video decoder)may form a so-called “palette” to represent the video data of theparticular area. The palette may be expressed as a table of colors orpixel values representing the video data of the particular area (e.g., agiven block). For example, the palette may include the most dominantpixel values in the given block. In some cases, the most dominant pixelvalues may include the one or more pixel values that occur mostfrequently within the block. Additionally, in some cases, a video codermay apply a threshold value to determine whether a pixel value is to beincluded as one of the most dominant pixel values in the block.According to various aspects of palette-based coding, the video codermay code index values indicative of one or more of the pixels values ofthe current block, instead of coding actual pixel values or theirresiduals for a current block of video data. In the context ofpalette-based coding, the index values indicate respective entries inthe palette that are used to represent individual pixel values of thecurrent block.

For example, the video encoder may encode a block of video data bydetermining the palette for the block (e.g., coding the paletteexplicitly, predicting the palette, or a combination thereof), locatingan entry in the palette to represent one or more of the pixel values,and encoding the block with index values that indicate the entry in thepalette used to represent the pixel values of the block. In someexamples, the video encoder may signal the palette and/or the indexvalues in an encoded bitstream. In turn, the video decoder may obtain,from an encoded bitstream, a palette for a block, as well as indexvalues for the individual pixels of the block. The video decoder mayrelate the index values of the pixels to entries of the palette toreconstruct the various pixel values of the block.

Palette-based coding of video data has been described in detail above.The basic idea of palette-based coding is that, for each CU, a paletteis derived which comprises (and may consist of) the most dominant pixelvalues in the current CU. The size and the elements of the palette arefirst transmitted from a video encoder to a video decoder. After that,the pixel values in the CU are encoded according to a certain scanningorder. For each pixel location in the CU, a flag, e.g., palette flag, isfirst transmitted to indicate if the pixel value is included in thepalette (i.e., “run mode”) or not (i.e., “pixel mode”). In “run mode,”the palette index associated with the pixel location in the CU issignaled followed by a “run” of the pixel value. Neither palette flagnor the palette index needs to be transmitted for the following pixellocations that are covered by the “run” as they all have the same pixelvalue. In “pixel mode,” the pixel value is transmitted for the givenpixel location in the CU.

For each CU, a major color table is derived which consists of the mostdominant pixel values in the current CU. The size and the elements ofthe table are first transmitted. The size and/or the elements of themajor color table can be directly coded or predictively coded using thesize and/or the elements of the major color table in the neighboring CUs(e.g. above and/or left coded CU).

In some examples, each of the pixels in the current CU is mapped to onemajor color table index. For those pixels whose major color indexes donot exist, a special index (named ‘other index’) is assigned to them andthese pixels are called ‘escaped pixel’. The techniques of thisdisclosure focus on the coding method of major color indexes.

An ‘escape pixel’ can be coded using any existing entropy coding methodsuch as fixed length coding, unary coding, etc. A method to encode theescape values is using a left-shift operation depending on thequantization parameter (QP) value. That is, encode the most significantbits only, being the number of bits depending on the QP value. To thatend, a strategy used in the state-of-the-art is to use a table that mapseach QP to a number which is the right-shift to be applied to the pixelvalue.

The block of major color index is coded line by line. For each line, aline mode is chosen from ‘horizontal’, ‘vertical’, and ‘normal’. If the‘horizontal’ line mode is chosen, all of the indexes in the current lineare the same as the left most index of the left most index in the aboveline. If the ‘vertical’ line mode is chosen, the entire line is copiedfrom the one line above. If the ‘normal’ mode is selected, the indexeswithin the current line are coded one by one. For each index in thiscase, a syntax element is used to indicate whether the index can becopied from the collocated index in the above line (‘copy_from_top’), orfrom the index's left neighbor index (‘copy_from_left’), or no copy ispossible (‘no_copy’). If no copy is possible, the pixel is codeddirectly.

The examples above are intended to provide a general description ofpalette-based coding. In various examples, the techniques described inthis disclosure may include techniques for various combinations of oneor more of signaling encoded video data formed by palette-based codingmodes, transmitting palettes, predicting palettes, deriving palettes, ordecoding video data from palette-based coding maps and other syntaxelements. Such techniques may improve video coding efficiency, e.g.,requiring fewer bits to represent screen generated content.

This disclosure describes various techniques related to palette-basedvideo coding. In some aspects, this disclosure is directed to bypassingcoding a map of index values for a video block when a palette associatedwith the video block only includes one entry or color, and the videoblock does not include any escape pixels. In some aspects, thisdisclosure is directed to deriving a palette “error limit,” whichindicates a maximum deviation from a fixed pixel value that may beincluded within the corresponding palette, using a mapping table thatstores a relationship between quantization parameter values and paletteerror limits. Some aspects of this disclosure are directed to deriving aquantization parameter (QP) for quantization and dequantization of anescape pixel (or the associated prediction error), using quantizationparameters used for traditional coefficient coding in a correspondingcolor channel. Additionally, this disclosure describes the use of a flagto indicate whether a pixel that is included in a palette-based codedarea is an escape pixel. Aspects of this disclosure also describetechniques to quantize an escape pixel value, such as techniques thatemploy a right-shift operation.

In some aspects, this disclosure is directed to quantizing escape pixelsof a palette-coded block according to a mapping function (e.g., a shiftfunction) based on a quantization parameter value for the escape pixel.In some aspects, this disclosure is directed to using a flag to indicateand/or determine whether a pixel of a palette-coded block is an escapepixel.

In some examples of this disclosure, the techniques for palette-basedcoding of video data may be used with one or more other codingtechniques, such as techniques for inter-predictive coding orintra-predictive coding of video data. For example, as described ingreater detail below, an encoder or decoder, or combined encoder-decoder(codec), may be configured to perform inter- and intra-predictivecoding, as well as palette-based coding. In some examples, thepalette-based coding techniques may be configured for use in one or morecoding modes of High Efficiency Video Coding (HEVC). In other examples,the palette-based coding techniques can be used independently or as partof other existing or future systems or standards.

High Efficiency Video Coding (HEVC) is a new video coding standarddeveloped by the Joint Collaboration Team on Video Coding (JCT-VC) ofITU-T Video Coding Experts Group (VCEG) and ISO/IEC Motion PictureExperts Group (MPEG). A recent draft of the HEVC standard, referred toas “HEVC Draft 10” of “WD10,” is described in document JCTVC-L1003v34,Bross et al., “High Efficiency Video Coding (HEVC) Text SpecificationDraft 10 (for FDIS & Last Call),” Joint Collaborative Team on VideoCoding (JCT-VC) of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11, 12^(th)Meeting: Geneva, CH, 14-23 Jan. 2013, available from:http://phenix.int-evey.fr/jct/doc_end_user/documents/12_Geneva/wg11/JCTVC-L1003-v34.zip.The finalized HEVC standard document is published as “ITU-T H.265,SERIES H: AUDIOVISUAL AND MULTIMEDIA SYSTEMS Infrastructure ofaudiovisual services—Coding of moving video—High efficiency videocoding,” Telecommunication Standardization Sector of InternationalTelecommunication Union (ITU), April 2013.

With respect to the HEVC framework, as an example, the palette-basedcoding techniques may be configured to be used as a coding unit (CU)mode. In other examples, the palette-based coding techniques may beconfigured to be used as a prediction unit (PU) mode in the framework ofHEVC. Accordingly, all of the following disclosed processes described inthe context of a CU mode may, additionally or alternatively, apply toPU. However, these HEVC-based examples should not be considered arestriction or limitation of the palette-based coding techniquesdescribed herein, as such techniques may be applied to workindependently or as part of other existing or yet to be developedsystems/standards. In these cases, the unit for palette coding can besquare blocks, rectangular blocks or even regions of non-rectangularshape.

FIG. 1 is a block diagram illustrating an example video coding system 10that may utilize the techniques of this disclosure. As used herein, theterm “video coder” refers generically to both video encoders and videodecoders. In this disclosure, the terms “video coding” or “coding” mayrefer generically to video encoding or video decoding. Video encoder 20and video decoder 30 of video coding system 10 represent examples ofdevices that may be configured to perform techniques for palette-basedvideo coding in accordance with various examples described in thisdisclosure. For example, video encoder 20 and video decoder 30 may beconfigured to selectively code various blocks of video data, such as CUsor PUs in HEVC coding, using either palette-based coding or non-palettebased coding. Non-palette based coding modes may refer to variousinter-predictive temporal coding modes or intra-predictive spatialcoding modes, such as the various coding modes specified by HEVC Draft10.

As shown in FIG. 1, video coding system 10 includes a source device 12and a destination device 14. Source device 12 generates encoded videodata. Accordingly, source device 12 may be referred to as a videoencoding device or a video encoding apparatus. Destination device 14 maydecode the encoded video data generated by source device 12.Accordingly, destination device 14 may be referred to as a videodecoding device or a video decoding apparatus. Source device 12 anddestination device 14 may be examples of video coding devices or videocoding apparatuses.

Source device 12 and destination device 14 may comprise a wide range ofdevices, including desktop computers, mobile computing devices, notebook(e.g., laptop) computers, tablet computers, set-top boxes, telephonehandsets such as so-called “smart” phones, televisions, cameras, displaydevices, digital media players, video gaming consoles, in-car computers,or the like.

Destination device 14 may receive encoded video data from source device12 via a channel 16. Channel 16 may comprise one or more media ordevices capable of moving the encoded video data from source device 12to destination device 14. In one example, channel 16 may comprise one ormore communication media that enable source device 12 to transmitencoded video data directly to destination device 14 in real-time. Inthis example, source device 12 may modulate the encoded video dataaccording to a communication standard, such as a wireless communicationprotocol, and may transmit the modulated video data to destinationdevice 14. The one or more communication media may include wirelessand/or wired communication media, such as a radio frequency (RF)spectrum or one or more physical transmission lines. The one or morecommunication media may form part of a packet-based network, such as alocal area network, a wide-area network, or a global network (e.g., theInternet). The one or more communication media may include routers,switches, base stations, or other equipment that facilitatecommunication from source device 12 to destination device 14.

In another example, channel 16 may include a storage medium that storesencoded video data generated by source device 12. In this example,destination device 14 may access the storage medium via disk access orcard access. The storage medium may include a variety oflocally-accessed data storage media such as Blu-ray discs, DVDs,CD-ROMs, flash memory, or other suitable digital storage media forstoring encoded video data.

In a further example, channel 16 may include a file server or anotherintermediate storage device that stores encoded video data generated bysource device 12. In this example, destination device 14 may accessencoded video data stored at the file server or other intermediatestorage device via streaming or download. The file server may be a typeof server capable of storing encoded video data and transmitting theencoded video data to destination device 14. Example file serversinclude web servers (e.g., for a website), file transfer protocol (FTP)servers, network attached storage (NAS) devices, and local disk drives.

Destination device 14 may access the encoded video data through astandard data connection, such as an Internet connection. Example typesof data connections may include wireless channels (e.g., Wi-Ficonnections), wired connections (e.g., DSL, cable modem, etc.), orcombinations of both that are suitable for accessing encoded video datastored on a file server. The transmission of encoded video data from thefile server may be a streaming transmission, a download transmission, ora combination of both.

The techniques of this disclosure are not limited to wirelessapplications or settings. The techniques may be applied to video codingin support of a variety of multimedia applications, such as over-the-airtelevision broadcasts, cable television transmissions, satellitetelevision transmissions, streaming video transmissions, e.g., via theInternet, encoding of video data for storage on a data storage medium,decoding of video data stored on a data storage medium, or otherapplications. In some examples, video coding system 10 may be configuredto support one-way or two-way video transmission to support applicationssuch as video streaming, video playback, video broadcasting, and/orvideo telephony.

Video coding system 10 illustrated in FIG. 1 is merely an example andthe techniques of this disclosure may apply to video coding settings(e.g., video encoding or video decoding) that do not necessarily includeany data communication between the encoding and decoding devices. Inother examples, data is retrieved from a local memory, streamed over anetwork, or the like. A video encoding device may encode and store datato memory, and/or a video decoding device may retrieve and decode datafrom memory. In many examples, the encoding and decoding is performed bydevices that do not communicate with one another, but simply encode datato memory and/or retrieve and decode data from memory.

In the example of FIG. 1, source device 12 includes a video source 18, avideo encoder 20, and an output interface 22. In some examples, outputinterface 22 may include a modulator/demodulator (modem) and/or atransmitter. Video source 18 may include a video capture device, e.g., avideo camera, a video archive containing previously-captured video data,a video feed interface to receive video data from a video contentprovider, and/or a computer graphics system for generating video data,or a combination of such sources of video data.

Video encoder 20 may encode video data from video source 18. In someexamples, source device 12 directly transmits the encoded video data todestination device 14 via output interface 22. In other examples, theencoded video data may also be stored onto a storage medium or a fileserver for later access by destination device 14 for decoding and/orplayback.

In the example of FIG. 1, destination device 14 includes an inputinterface 28, a video decoder 30, and a display device 32. In someexamples, input interface 28 includes a receiver and/or a modem. Inputinterface 28 may receive encoded video data over channel 16. Displaydevice 32 may be integrated with or may be external to destinationdevice 14. In general, display device 32 displays decoded video data.Display device 32 may comprise a variety of display devices, such as aliquid crystal display (LCD), a plasma display, an organic lightemitting diode (OLED) display, or another type of display device.

This disclosure may generally refer to video encoder 20 “signaling” or“transmitting” certain information to another device, such as videodecoder 30. The term “signaling” or “transmitting” may generally referto the communication of syntax elements and/or other data used to decodethe compressed video data. Such communication may occur in real- ornear-real-time. Alternately, such communication may occur over a span oftime, such as might occur when storing syntax elements to acomputer-readable storage medium in an encoded bitstream at the time ofencoding, which then may be retrieved by a decoding device at any timeafter being stored to this medium. Thus, while video decoder 30 may bereferred to as “receiving” certain information, the receiving ofinformation does not necessarily occur in real- or near-real-time andmay be retrieved from a medium at some time after storage.

Video encoder 20 and video decoder 30 each may be implemented as any ofa variety of suitable circuitry, such as one or more microprocessors,digital signal processors (DSPs), application-specific integratedcircuits (ASICs), field-programmable gate arrays (FPGAs), discretelogic, hardware, or any combinations thereof. If the techniques areimplemented partially in software, a device may store instructions forthe software in a suitable, non-transitory computer-readable storagemedium and may execute the instructions in hardware using one or moreprocessors to perform the techniques of this disclosure. Any of theforegoing (including hardware, software, a combination of hardware andsoftware, etc.) may be considered to be one or more processors. Each ofvideo encoder 20 and video decoder 30 may be included in one or moreencoders or decoders, either of which may be integrated as part of acombined encoder/decoder (CODEC) in a respective device.

In some examples, video encoder 20 and video decoder 30 operateaccording to a video compression standard, such as HEVC standardmentioned above, and described in HEVC Draft 10. In addition to the baseHEVC standard, there are ongoing efforts to produce scalable videocoding, multiview video coding, and 3D coding extensions for HEVC. Inaddition, palette-based coding modes, e.g., as described in thisdisclosure, may be provided for extension of the HEVC standard. In someexamples, the techniques described in this disclosure for palette-basedcoding may be applied to encoders and decoders configured to operationaccording to other video coding standards, such as theITU-T-H.264/AVCstandard or future standards. Accordingly, application of apalette-based coding mode for coding of coding units (CUs) or predictionunits (PUs) in an HEVC codec is described for purposes of example.

In HEVC and other video coding standards, a video sequence typicallyincludes a series of pictures. Pictures may also be referred to as“frames.” A picture may include three sample arrays, denoted S_(L),S_(Cb) and S_(Cr). S_(L) is a two-dimensional array (i.e., a block) ofluma samples. S_(Cb) is a two-dimensional array of Cb chrominancesamples. S_(Cr) is a two-dimensional array of Cr chrominance samples.Chrominance samples may also be referred to herein as “chroma” samples.In other instances, a picture may be monochrome and may only include anarray of luma samples.

To generate an encoded representation of a picture, video encoder 20 maygenerate a set of coding tree units (CTUs). Each of the CTUs may be acoding tree block of luma samples, two corresponding coding tree blocksof chroma samples, and syntax structures used to code the samples of thecoding tree blocks. A coding tree block may be an N×N block of samples.A CTU may also be referred to as a “tree block” or a “largest codingunit” (LCU). The CTUs of HEVC may be broadly analogous to themacroblocks of other standards, such as H.264/AVC. However, a CTU is notnecessarily limited to a particular size and may include one or morecoding units (CUs). A slice may include an integer number of CTUsordered consecutively in the raster scan. A coded slice may comprise aslice header and slice data. The slice header of a slice may be a syntaxstructure that includes syntax elements that provide information aboutthe slice. The slice data may include coded CTUs of the slice.

This disclosure may use the term “video unit” or “video block” or“block” to refer to one or more sample blocks and syntax structures usedto code samples of the one or more blocks of samples. Example types ofvideo units or blocks may include CTUs, CUs, PUs, transform units (TUs),macroblocks, macroblock partitions, and so on. In some contexts,discussion of PUs may be interchanged with discussion of macroblocks ormacroblock partitions.

To generate a coded CTU, video encoder 20 may recursively performquad-tree partitioning on the coding tree blocks of a CTU to divide thecoding tree blocks into coding blocks, hence the name “coding treeunits.” A coding block is an N×N block of samples. A CU may be a codingblock of luma samples and two corresponding coding blocks of chromasamples of a picture that has a luma sample array, a Cb sample array anda Cr sample array, and syntax structures used to code the samples of thecoding blocks. Video encoder 20 may partition a coding block of a CUinto one or more prediction blocks. A prediction block may be arectangular (i.e., square or non-square) block of samples on which thesame prediction is applied. A prediction unit (PU) of a CU may be aprediction block of luma samples, two corresponding prediction blocks ofchroma samples of a picture, and syntax structures used to predict theprediction block samples. Video encoder 20 may generate predictive luma,Cb and Cr blocks for luma, Cb and Cr prediction blocks of each PU of theCU.

Video encoder 20 may use intra prediction or inter prediction togenerate the predictive blocks for a PU. If video encoder 20 uses intraprediction to generate the predictive blocks of a PU, video encoder 20may generate the predictive blocks of the PU based on decoded samples ofthe picture associated with the PU.

If video encoder 20 uses inter prediction to generate the predictiveblocks of a PU, video encoder 20 may generate the predictive blocks ofthe PU based on decoded samples of one or more pictures other than thepicture associated with the PU. Video encoder 20 may use uni-predictionor bi-prediction to generate the predictive blocks of a PU. When videoencoder 20 uses uni-prediction to generate the predictive blocks for aPU, the PU may have a single motion vector (MV). When video encoder 20uses bi-prediction to generate the predictive blocks for a PU, the PUmay have two MVs.

After video encoder 20 generates predictive blocks (e.g., predictiveluma, Cb and Cr blocks) for one or more PUs of a CU, video encoder 20may generate residual blocks for the CU. Each sample in a residual blockof the CU may indicate a difference between a sample in a predictiveblock of a PU of the CU and a corresponding sample in a coding block ofthe CU. For example, video encoder 20 may generate a luma residual blockfor the CU. Each sample in the CU's luma residual block indicates adifference between a luma sample in one of the CU's predictive lumablocks and a corresponding sample in the CU's original luma codingblock. In addition, video encoder 20 may generate a Cb residual blockfor the CU. Each sample in the CU's Cb residual block may indicate adifference between a Cb sample in one of the CU's predictive Cb blocksand a corresponding sample in the CU's original Cb coding block. Videoencoder 20 may also generate a Cr residual block for the CU. Each samplein the CU's Cr residual block may indicate a difference between a Crsample in one of the CU's predictive Cr blocks and a correspondingsample in the CU's original Cr coding block.

Furthermore, video encoder 20 may use quad-tree partitioning todecompose the residual blocks (e.g., luma, Cb and Cr residual blocks) ofa CU into one or more transform blocks (e.g., luma, Cb and Cr transformblocks). A transform block may be a rectangular block of samples onwhich the same transform is applied. A transform unit (TU) of a CU maybe a transform block of luma samples, two corresponding transform blocksof chroma samples, and syntax structures used to transform the transformblock samples. Thus, each TU of a CU may be associated with a lumatransform block, a Cb transform block, and a Cr transform block. Theluma transform block associated with the TU may be a sub-block of theCU's luma residual block. The Cb transform block may be a sub-block ofthe CU's Cb residual block. The Cr transform block may be a sub-block ofthe CU's Cr residual block.

Video encoder 20 may apply one or more transforms to a transform blockto generate a coefficient block for a TU. A coefficient block may be atwo-dimensional array of transform coefficients. A transform coefficientmay be a scalar quantity. For example, video encoder 20 may apply one ormore transforms to a luma transform block of a TU to generate a lumacoefficient block for the TU. Video encoder 20 may apply one or moretransforms to a Cb transform block of a TU to generate a Cb coefficientblock for the TU. Video encoder 20 may apply one or more transforms to aCr transform block of a TU to generate a Cr coefficient block for theTU.

After generating a coefficient block (e.g., a luma coefficient block, aCb coefficient block or a Cr coefficient block), video encoder 20 mayquantize the coefficient block. Quantization generally refers to aprocess in which transform coefficients are quantized to possibly reducethe amount of data used to represent the transform coefficients,providing further compression. After video encoder 20 quantizes acoefficient block, video encoder 20 may entropy encoding syntax elementsindicating the quantized transform coefficients. For example, videoencoder 20 may perform Context-Adaptive Binary Arithmetic Coding (CABAC)on the syntax elements indicating the quantized transform coefficients.Video encoder 20 may output the entropy-encoded syntax elements in abitstream. The bitstream may also include syntax elements that are notentropy encoded.

Video encoder 20 may output a bitstream that includes theentropy-encoded syntax elements. The bitstream may include a sequence ofbits that forms a representation of coded pictures and associated data.The bitstream may comprise a sequence of network abstraction layer (NAL)units. Each of the NAL units includes a NAL unit header and encapsulatesa raw byte sequence payload (RBSP). The NAL unit header may include asyntax element that indicates a NAL unit type code. The NAL unit typecode specified by the NAL unit header of a NAL unit indicates the typeof the NAL unit. A RBSP may be a syntax structure containing an integernumber of bytes that is encapsulated within a NAL unit. In someinstances, an RBSP includes zero bits.

Different types of NAL units may encapsulate different types of RBSPs.For example, a first type of NAL unit may encapsulate an RBSP for apicture parameter set (PPS), a second type of NAL unit may encapsulatean RBSP for a coded slice, a third type of NAL unit may encapsulate anRBSP for supplemental enhancement information (SEI), and so on. NALunits that encapsulate RBSPs for video coding data (as opposed to RBSPsfor parameter sets and SEI messages) may be referred to as video codinglayer (VCL) NAL units.

Video decoder 30 may receive a bitstream generated by video encoder 20.In addition, video decoder 30 may obtain syntax elements from thebitstream. For example, video decoder 30 may parse the bitstream todecode syntax elements from the bitstream. Video decoder 30 mayreconstruct the pictures of the video data based at least in part on thesyntax elements obtained (e.g., decoded) from the bitstream. The processto reconstruct the video data may be generally reciprocal to the processperformed by video encoder 20. For instance, video decoder 30 may useMVs of PUs to determine predictive sample blocks (i.e., predictiveblocks) for the PUs of a current CU. In addition, video decoder 30 mayinverse quantize transform coefficient blocks associated with TUs of thecurrent CU. Video decoder 30 may perform inverse transforms on thetransform coefficient blocks to reconstruct transform blocks associatedwith the TUs of the current CU. Video decoder 30 may reconstruct thecoding blocks of the current CU by adding the samples of the predictivesample blocks for PUs of the current CU to corresponding samples of thetransform blocks of the TUs of the current CU. By reconstructing thecoding blocks for each CU of a picture, video decoder 30 may reconstructthe picture.

In some examples, video encoder 20 and video decoder 30 may beconfigured to perform palette-based coding. For example, in palettebased coding, rather than performing the intra-predictive orinter-predictive coding techniques described above, video encoder 20 andvideo decoder 30 may code a so-called palette as a table of colors orpixel values representing the video data of a particular area (e.g., agiven block). In this way, rather than coding actual pixel values ortheir residuals for a current block of video data, the video coder maycode index values for one or more of the pixels values of the currentblock, where the index values indicate entries in the palette that areused to represent the pixel values of the current block.

For example, video encoder 20 may encode a block of video data bydetermining a palette for the block, locating an entry in the palettehaving a value representative of the value of one or more individualpixels of the block, and encoding the block with index values thatindicate the entry in the palette used to represent the one or moreindividual pixel values of the block. Additionally, video encoder 20 maysignal the index values in an encoded bitstream. In turn, a videodecoding device (e.g., video decoder 30) may obtain, from the encodedbitstream, the palette for a block, as well as index values used fordetermining the various individual pixels of the block using thepalette. Video decoder 30 may match the index values of the individualpixels to entries of the palette to reconstruct the pixel values of theblock. In instances where the index value associated with an individualpixel does not match any index value of the corresponding palette forthe block, video decoder 30 may identify such a pixel as an escapepixel, for the purposes of palette-based coding.

In another example, video encoder 20 may encode a block of video dataaccording to the following operations. Video encoder 20 may determineprediction residual values for individual pixels of the block, determinea palette for the block, and locate an entry (e.g., index value) in thepalette having a value representative of the value of one or more of theprediction residual values of the individual pixels. Additionally, videoencoder 20 may encode the block with index values that indicate theentry in the palette used to represent the corresponding predictionresidual value for each individual pixel of the block. Video decoder 30may obtain, from an encoded bitstream signaled by source device 12, apalette for a block, as well as index values for the prediction residualvalues corresponding to the individual pixels of the block. Asdescribed, the index values may correspond to entries in the paletteassociated with the current block. In turn, video decoder 30 may relatethe index values of the prediction residual values to entries of thepalette to reconstruct the prediction residual values of the block. Theprediction residual values may be added to the prediction values (forexample, obtained using intra or inter prediction) to reconstruct thepixel values of the block.

As described in more detail below, the basic idea of palette-basedcoding is that, for a given block of video data to be coded, videoencoder 20 may derive a palette that includes the most dominant pixelvalues in the current block. For instance, the palette may refer to anumber of pixel values which are determined or assumed to be dominantand/or representative for the current CU. Video encoder 20 may firsttransmit the size and the elements of the palette to video decoder 30.Additionally, video encoder 20 may encode the pixel values in the givenblock according to a certain scanning order. For each pixel included inthe given block, video encoder 20 may signal the index value that mapsthe pixel value to a corresponding entry in the palette. If the pixelvalue is not included in the palette (i.e., no palette entry exists thatspecifies a particular pixel value of the palette-coded block), thensuch a pixel is defined as an “escape pixel.” In accordance withpalette-based coding, video encoder 20 may encode and signal an indexvalue that is reserved for an escape pixel. In some examples, videoencoder 20 may also encode and signal the pixel value or a residualvalue (or quantized versions thereof) for an escape pixel included inthe given block.

Upon receiving the encoded video bitstream signaled by video encoder 20,video decoder 30 may first determine the palette based on theinformation received from video encoder 20. Video decoder 30 may thenmap the received index values associated with the pixel locations in thegiven block to entries of the palette to reconstruct the pixel values ofthe given block. In some instances, video decoder 30 may determine thata pixel of a palette-coded block is an escape pixel, such as bydetermining that the pixel is palette-coded with an index value reservedfor escape pixels. In instances where video decoder 30 identifies anescape pixel in a palette-coded block, video decoder 30 may receive thepixel value or a residual value (or quantized versions thereof) for anescape pixel included in the given block. Video decoder 30 mayreconstruct the palette-coded block by mapping the individual pixelvalues to the corresponding palette entries, and by using the pixelvalue or residual value (or quantized versions thereof) to reconstructany escape pixels included in the palette-coded block.

Palette-based coding may introduce an amount of signaling overhead. Forexample, a number of bits may be needed to signal characteristics of apalette, such as a size of the palette, as well as the palette itself.In addition, a number of bits may be needed to signal index values forthe pixels of the block. For instance, according to existingpalette-based coding techniques, even in cases where the palette is oflimited size (e.g., a palette that includes just one entry), and a blockdoes not include any escape pixels, video encoder 20 may still signalpalette indices for the pixel values of the block, which will all be thesame index value identifying the one entry in the palette, on aline-by-line basis for the video block. Additionally, existingpalette-based coding techniques introduce signaling overhead in terms ofsignaling an index value to indicate an escape pixel, and then signalingthe pixel value or residual value (or quantized versions thereof) forthe escape pixel.

The techniques of this disclosure may, in some examples, reduce thenumber of bits needed to signal such information. For example, certaintechniques described herein are directed to bypassing coding (e.g.,encoding and/or decoding) of a map of index values for one or morepixels of a block if the block satisfies a particular set of conditionswith respect to palette-based coding. In other examples, certaintechniques described herein are generally directed to signaling lessbits of data (e.g., a one-bit flag instead of a five-bit index value) toindicate that a given pixel is an escape pixel with respect to thepalette for the current block. Various techniques of this disclosure arealso directed to determining a range of pixel values that may beincluded in a palette for a given block. The range of pixel values thata palette may include is referred to herein as the palette's “errorlimit,” and various techniques of this disclosure are directed todetermining a palette's error limit based on the quantization parameter(QP) of the block associated with the palette.

Other aspects of this disclosure are directed to deriving quantizedescape values. For instance, some of these aspects are directed totechniques for defining one or more quantization parameters with whichto quantize an escape pixel. Still other aspects of this disclosure aredirected to applying particular functions (e.g., a right-shift function)in quantizing an escape pixel value. In this manner, various aspects ofthis disclosure provide potential advantages, such as reducing bitoverhead and mitigating resource usage, while maintaining picturequality and accuracy.

As described above, video encoder 20 may apply techniques of thisdisclosure to bypass encoding and signaling of a palette index forvarious individual pixels of a block under certain circumstances.According to aspects of this disclosure, video encoder 20 may bypassencoding and signaling of the palette index for a palette-coded block ifvideo encoder 20 determines that all pixels of the block are of the samecolor. For instance, video encoder 20 may determine that a palette-codedCU of a picture is a “single color CU” and may bypass encoding andsignaling of the palette index for the single color CU.

More specifically, video encoder 20 may determine whether apalette-coded CU is a single color CU if the current CU satisfies twoconditions. The first condition that video encoder 20 may use indetermining whether the palette-coded CU is a single color CU is whetherthe size of the corresponding palette is equal to one (1). If the sizeof the palette is equal to one, then video encoder 20 may determine thatthe first condition is met with respect to the palette-coded CU being asingle color CU. More specifically, if the palette size is one, thenvideo decoder 20 may determine that the palette includes only one colorthat corresponds to (non-escape) pixels of the palette-coded CU. In someexamples where the palette size is one, video encoder 20 may determinethat the only index value associated with the palette is zero (0).

If video encoder 20 determines that the palette size for thepalette-coded CU is 1 (i.e., that the first condition is met), thenvideo encoder 20 may determine whether the palette-coded CU meets asecond condition to be a single color CU. The second condition thatvideo decoder 30 may use in determining whether the palette-coded CU isa single color CU is that the palette-coded CU does not include anyescape pixels. If the palette-coded CU includes at least one escapepixel, then video encoder 20 may determine that, even though thecorresponding palette indicates only one color with respect to thepalette-coded CU, the palette-coded CU includes pixels of two or morecolors. For instance, the palette-coded CU may include at least onepixel that has the color indicated in the palette, and at least oneescape pixel that has a different color.

If video encoder 20 determines that the palette-coded CU satisfies bothof the conditions described above, then video encoder 20 may determinethat the palette-coded CU is a single color CU. More specifically, ifthe palette-coded CU is associated with a single-entry palette (shown bythe palette size of one), and the palette-coded CU does not include anyescape pixels, then video encoder 20 may determine that all individualpixels of the palette-coded CU are of the same color (i.e., the colorindicated by the single entry of the corresponding palette). In variousimplementations, video encoder 20 may apply the single color CUidentification techniques described above with respect to a single colorcomponent basis with respect to the palette, or to a combined index thatindicates more than one color component.

Responsive to determining that the palette-coded CU includes pixels ofonly one color (i.e., that the CU is a single color CU), video encoder20 may implement techniques of this disclosure to bypass encoding andsignaling the map of palette index values for the pixels of the singlecolor CU. By bypassing the encoding and signaling of the map of paletteindex values for the pixels of the CU, video encoder 20 may conservecomputing resources and bandwidth that would otherwise be expended forencoding and signaling color information for the CU. Instead of encodingand signaling an index for each individual pixel of the single color CU,video encoder 20 may implement techniques of this disclosure to moreefficiently indicate (e.g., to video decoder 30), the color informationfor the entire set of pixels that make up the single color CU.

According to some examples of this disclosure, video encoder 20 mayencode and signal a flag to indicate whether video encoder 20 bypassedencoding (and signaling) of the palette index values on a line-by-linebasis for the single color CU. By encoding and signaling a one-bit flagfor the entire CU instead of individual index values for each pixel ofthe CU, video encoder 20 may conserve computing resources and signalingbandwidth in comparison to existing palette-based coding techniques.Moreover, video encoder 20 may maintain accuracy and quality of theencoded single color CU, because the single-entry palette signaled byvideo encoder 20 for the CU includes the color information for allindividual pixels of the CU. In various examples, video encoder 20 mayencode and signal the flag in various ways, such as in a sequenceparameter set (SPS), a picture parameter set (PPS), or a slice header.In various examples, video encoder 20 may encode and signal the flag ona per-CTU basis, a per-CU basis, or for a block of any block size, aswell.

In examples where video encoder 20 bypasses encoding and signaling ofthe palette index value for individual pixels of a single color blockthat is palette-coded, video decoder 30 may apply various techniques ofthis disclosure to reconstruct the single color block. In some examples,video decoder 30 may perform operations reciprocal to those describedabove with respect to video encoder 20 to determine that thepalette-coded block is a single color block. For instance, video decoder30 may determine that the palette for the current block has a size ofone, thereby determining that the block satisfies the first condition toqualify as a single color block. In various examples, video decoder 30may receive the palette in an encoded video bitstream from video encoder20, or may reconstruct the palette.

Additionally, video decoder 30 may determine that the block does notinclude any escape pixels, thereby determining that the block satisfiesthe second condition to qualify as a single color block. Based ondetermining that the size of the palette for the block is one (the firstcondition), and that the block does not include any escape pixels (thesecond condition), video decoder 30 may implement techniques of thisdisclosure to determine that the current block is a single color block.In this manner, video decoder 30 may implement techniques of thisdisclosure to reconstruct a palette-coded block accurately, whileconserving computing resources and bandwidth that would otherwise berequired to reconstruct the block by decoding a palette index on apixel-by-pixel basis.

In other examples, video decoder 30 may receive, in the encoded videobitstream, a flag that indicates whether video encoder 20 bypassedencoding and signaling of the palette index for a palette-coded block,in accordance with techniques of this disclosure. In cases where videodecoder 30 receives a flag indicating that video encoder 20 bypassedencoding and signaling of the map of palette index values for thepalette-coded block, video decoder 30 may implement techniques of thisdisclosure to determine that the current block is palette-coded, and isa single color block. More specifically, if the flag is enabled (e.g.,set to a value of one), video decoder 30 may determine that thepalette-coded block is a single color block. In turn, video decoder 30may implement techniques of this disclosure to reconstruct each pixel ofthe block according to the color information of the single entry in thepalette for the block. In this manner, video decoder 30 may implementtechniques of this disclosure to accurately reconstruct thepalette-coded block using a one-bit flag for the entire block, ratherthan using separate index values (of varying bitdepth) for eachindividual pixel of the block.

In another example, video encoder 20 may implement techniques of thisdisclosure to derive an error limit for a palette for a palette-codedblock. As used herein, the terms “error limit” or “palette error limit”may refer to the range of values (e.g., in terms of color information)that the entries of the palette can include. More specifically, thepalette error limit defines a minimum variation in color value thatdifferent palette entries bear, or must display. As described above, inorder to encode a block according to palette-based coding, video encoder20 may construct the corresponding palette to include color values thatoccur most frequently (on a pixel-by-pixel basis) within the block.

In constructing a palette, video encoder 20 may determine that thevarious entries of the palette must display a minimum variation from oneanother. More specifically, video encoder 20 may construct the paletteso that no two entries of the palette are sufficiently similar such thatthe two entries can be grouped as a single entry. If two possiblepalette entries are within the palette error limit, video encoder 20 mayuse one of the two entries to represent both entries in the palette.

However, if video encoder 20 determines that two entries (which occurcommonly in the block) differ by at least the palette error limit, thenvideo encoder 20 may include both entries in the palette. In exampleswhere the entries are represented by three color components, videoencoder 20 may include both entries in the palette if at the entriesdiffer by at least the palette error limit with respect to at least oneof the color components. For instance, if the palette error limit is setto a value of five (5), video encoder 20 may determine whether toinclude both entries in the palette (assuming both entries occurcommonly enough in the block), based on the following Booleanexpression: abs(A1−B1)>5∥abs(A2−B2)>5∥abs(A3−B3)>5, where “abs”represents a difference between color component parameters.

As described, video encoder 20 may construct a palette by clusteringcommonly-occurring (or relatively commonly-occurring) pixel values ofthe block into entries of the palette. Video encoder 20 may select thecommonly-occurring pixel values such that the pixel values display aminimum variation, in terms of color information. In turn, the minimumvariation between pixel values within the selected set ofcommonly-occurring pixel values may form the error limit of thecorresponding palette. It will be appreciated that, while the paletteerror limit may include several pixel values, the palette may notnecessarily include every pair of pixel values that differ by at leastthe palette error limit. Thus, the same palette error limit may apply topalettes of varying sizes. Video encoder 20 may use the palette errorlimit in making determinations as to the color values that are to beincluded in the palette.

Video encoder 20 may implement techniques of this disclosure to definethe error limit for a palette. According to various aspects of thisdisclosure, video encoder 20 may determine the palette error limit basedon the quantization parameter (QP) for the palette-coded block. Invarious examples, video encoder 20 may determine that the palette errorlimit is directly proportional to the QP value for the correspondingblock. More specifically, in these examples, video encoder 20 may assigna larger error limit for a palette for a block that is quantized with agreater QP value, and a smaller error limit for a palette for a blockthat is quantized with a lesser QP value.

Thus, video encoder 20 may define palettes that require a greatervariation between pixel values for blocks that are quantized withgreater QP values, and may define palettes that require a lesservariation between pixel values for blocks that are quantized withgreater QP values. Additionally, video encoder 20 may generate and/orstore a table (e.g., a mapping table or look-up table) to reflect therelationship between each QP value and the corresponding palette errorlimit. In this manner, video encoder 20 may implement various techniquesof this disclosure to improve computational efficiency by using a tableto store the relationship between each QP value and the correspondingerror limit. More specifically, by using a table to store therelationship between the QP values and the corresponding palette errorlimit, video encoder 20 may implement the techniques described herein toprovide improved efficiency over the relatively computationallyexpensive techniques of solving a function for each palette to derivethe corresponding palette error limit. Thus, video encoder 20 maycustomize the palette (in accordance with the palette's error limit)based on the QP value with which the corresponding block is quantized,thereby determining the contents of the palette for a block based on theblock's QP value, in accordance with various aspects of this disclosure.

Video encoder 20 may, in some examples, implement various techniques ofthis disclosure for quantized escape pixel derivation. Morespecifically, video encoder 20 may implement the techniques to definethe quantization value of the QP for an escape pixel. For example,according to palette-based coding techniques, if video encoder 20detects an escape pixel in a palette-coded block, video encoder 20 mayencode and signal the pixel value, or a prediction error thereof,because the corresponding palette does not include any entries for theescape pixel. Additionally, to conserve signaling bandwidth, videoencoder 20 may quantize the encoded pixel value of the escape pixelprior to signaling.

According to existing palette-based coding techniques, no quantizationvalue (QP value) was defined for quantizing an escape pixel. Videoencoder 20 may implement techniques of this disclosure to define the QPvalue for quantizing an escape pixel. More specifically, video encoder20 may define the QP value for an escape pixel as equal to the QP valuefor traditional coefficient encoding within the same color channel(e.g., luma (Y), chroma (U, Cr), or chroma (V, Cb)). In one example,video encoder 20 may define the QP value for an escape pixel as equal tothe QP value for traditional coefficient encoding within the same colorchannel, and within the same quantization group. Thus, video encoder 20may quantize all escape pixels according to a single QP value within agiven channel. Additionally, as video encoder 20 may define the QP valuefor all escape pixels only within a single channel, video encoder 20 mayuse different QP values for quantizing escape pixels with respect todifferent channels.

Video decoder 30 may perform reciprocal operations to those describedabove, to dequantize escape pixels in accordance with various techniquesof this disclosure. For instance, video decoder 30 may dequantize allescape pixels of a single channel using the same QP value, based oninformation received in the encoded video bitstream from video encoder20. More specifically, in accordance with aspects of this disclosure,video decoder 30 may dequantize any escape pixels (or predictionerrors/residual values thereof) communicated over a particular channelusing a QP value that is determined based on the QP value fortraditional transform coefficient dequantization for blocks communicatedover the current channel. In some examples, video decoder 30 mayimplement the techniques of this disclosure to dequantize escape pixelscommunicated over different channels using different QP values, based onthe QP value for traditional transform coefficient coding beingdifferent among the different channels.

In this manner, video encoder 20 and video decoder 30 may implement thetechniques described herein to define and apply a single QP value (toquantize and/or dequantize) all escape pixels communicated over aparticular channel. Thus, video encoder 20 and video decoder 30 mayapply aspects of this disclosure to define a QP value for escape pixelsdetected through palette-based coding, where existing palette-basedcoding techniques did not explicitly define a QP value for escapepixels.

Additionally, video encoder 20 and/or video decoder 30 may implementother techniques of this disclosure to use a flag to indicate and/ordetect the inclusion of an escape pixel in a palette-coded block.According to existing palette-based coding techniques, escape pixels maybe signaled and detected using a “reserved” palette index value. Forinstance, according to the existing palette-based coding techniques, thereserved palette index value that indicates an escape pixel may be 32.More specifically, the palette index value of 32 may be used for allescape pixels, regardless of whether two escape pixels have differentpixel values. Thus, according to the existing palette-based codingtechniques, video coding devices may use a five-bit value (of 32) foreach escape pixel of a palette-coded block.

Video encoder 20 may implement techniques of this disclosure to conservecomputing resources (e.g., storage and memory) and reduce bandwidthconsumption, while maintaining picture precision with respect tosignaling an indication of an escape pixel in a palette-coded block. Forinstance, video encoder 20 may encode and signal a flag to indicatewhether a pixel in a palette-coded block is an escape pixel. Asdescribed herein, the flag, when enabled, may indicate that theassociated pixel is assigned a palette index referred to as “otherindex.” Video encoder 20 may use the “other index” status of the flag toreplace the palette index value of 32 that is traditionally used toindicate an escape pixel with respect to the palette. Thus, videoencoder 20 may encode and signal a one-bit flag instead of a five-bitindex value to indicate that a pixel of a palette-coded block is anescape pixel. In turn, when an escape pixel is indicated by the one-bitflag, video encoder 20 may encode and signal the pixel value (orresidual data thereof) of the escape pixel in the encoded videobitstream.

Video decoder 30 may also implement techniques of this disclosure to usea one-bit flag to determine that a pixel of a palette-coded block is anescape pixel. In various examples, video decoder 30 may performreciprocal operations with respect to the encoding and signalingoperations described above with respect to video encoder 20, to use theone-bit flag to identify an escape pixel in a palette-coded block. Forinstance, video decoder 30 may receive an enabled one-bit flagassociated with a pixel of a palette-coded block. Based on the one-bitflag being in the enabled state, video decoder 30 may determine that thecolor information for the associated pixel is not included in thepalette for the current block. In other words, video decoder 30 maydetermine that, if the received one-bit flag is enabled, the associatedpixel is an escape pixel. In this manner, video decoder 30 may implementthe techniques of this disclosure to reconstruct a palette-coded blockusing a one-bit flag to identify an escape pixel in the palette-codedblock. Thus, video decoder 30 may conserve computing resources (e.g.,storage and/or memory) and bandwidth requirements with respect toidentifying escape pixels in palette-coded blocks. Additionally, when anescape pixel is indicated by the one-bit flag, video decoder 30 mayreceive, in the encoded video bitstream, the color information (orresidual data thereof) corresponding to any identified escape pixels,and may reconstruct the palette-coded block accordingly.

Video encoder 20 and video decoder 30 may also implement techniques ofthis disclosure to quantize and dequantize the pixel values of escapepixels in accordance with palette-based coding. For instance, videoencoder 20 may conserve computing resources (e.g., memory usage,processor clock cycles, etc.) by quantizing the pixel values of escapepixels according to aspects of this disclosure. In some examples, videoencoder 20 may implement the techniques described herein to quantize theescape pixel values by substituting divide operations with shiftoperations (e.g., right-shift operations). More specifically, videoencoder 20 may determine the specific right-shift operation based on theQP value of the corresponding escape pixel.

For instance, video encoder 20 may form a table that maps the QP valueof each escape pixel to the amount of the right-shift to apply to thepixel value. Video encoder 20 may form the table to include 52 entries.For example, the 52-entry mapping table may provide a right-shift amountcorresponding to each possible QP value for a given escape pixel.Alternatively, video encoder 20 may apply a mapping operation todetermine the right-shift amount for each pixel, based on thecorresponding QP value entry in the table. The mapping function may bemore computationally efficient and may conserve memory requirements, incomparison to the 52-entry mapping table used according to existingquantization techniques for escape pixels according to palette-basedcoding. By deriving the right-shift value (operand) by solving afunction as described herein, video encoder 20 may eliminate the needfor video decoder 30 to store a 52-entry table, thereby enabling videodecoder 30 to dequantize escape pixels while reducing storagerequirements for the dequantization process.

In various examples, video encoder 20 may quantize an escape pixel bydetermining the right-shift amount for an escape pixel based on themapping operation described above, and applying a linear function to theescape pixel value, using the determined right-shift amount. An exampleof a linear function that video encoder 20 may apply to quantize anescape pixel is as follows:

Right_shift=a*((QP+b)>>c)+d,

where a, b, c, and d are all integer parameters. Additionally, the “>>”operator denotes the right-shift operation. In a specific result ofapplying the equation above, video encoder 20 may determine that theright-shift amount for an escape pixel value is three. The resultingright-shift operation may be expressed as Right_shift=(QP>>3)

Video decoder 30 may implement techniques of this disclosure to performreciprocal operations of those described above with respect to videoencoder 20, to dequantize a quantized escape pixel value. For instance,video decoder 30 may implement techniques of this disclosure tocalculate a shift amount (e.g., for a corresponding left-shiftoperation) based on a QP value in dequantizing the correspondingquantized escape pixel value. In this manner, video decoder 30 may alsoapply aspects of this disclosure to conserve computing resources byleveraging a mapping function instead of storing a 52-entry mappingtable.

As described above, video encoder 20 and/or video decoder 30 mayimplement various techniques of this disclosure, whether individually orin any combination and/or sequence, to provide improved codingefficiency with respect to palette-based coding, while maintainingpicture quality and data precision. Thus, the techniques describedherein may provide various potential advantages over existing techniquesof palette-based video coding. In specific examples, as described above,the techniques of this disclosure may enable video coding devices tomore efficiently encode and/or decode video data and reduce bandwidthconsumption, while maintaining accuracy of the video data.

In some examples, the techniques for palette-based coding of video datadescribed herein may be used with one or more other coding techniques,such as techniques for inter- or intra-predictive coding. For example,as described in greater detail below, an encoder or decoder, or combinedencoder-decoder (codec), may be configured to perform inter- andintra-predictive coding, as well as palette-based coding.

In various examples, this disclosure describes different aspects ofmajor color index coding techniques. It may be possible to combine partor all of the described methods.

An example of coding of index prediction direction is now described. Asstated above, for each index, there are three possible predictiondirections: ‘copy_from_top’, ‘copy_from_left’, and ‘no_copy’. Three codewords should be assigned to the three directions. For example, the codewords can be ‘0’, ‘01’, and ‘10’. In the case when the collocated pixelin the above line and the left neighboring pixel are the same, only twocode words may be necessary. For example, in this case, ‘0’ canrepresent ‘no copy’ and ‘1’ can represent copy from top or left.

As described above, in some cases, coding of the color index map for avideo block may be bypassed. If the number of major colors equals to oneand there is no ‘escape pixel,’ then the coding of index block can bebypassed. This principle can either be applied to each individual colorcomponent, or can be applied to the combine index which contain morethan one color component.

In another example, a flag (or other type of syntax element) can besignaled in the coded bitstream to indicate whether this feature ofbypassing index coding is used or not. For instance, a video encoder maysignal, in a bitstream that comprises a coded representation of videodata, a syntax element (e.g., the flag) to indicate whether or notbypassing index coding is used. Accordingly, a video decoder may obtain,from a bitstream, a syntax element that indicates whether or notbypassing index coding is used. The flag can be signaled in a SPS, PPS,slice header, or other structure, or per CTU or per CU or in any otherblock sizes.

Thus, in some examples, the video encoder may signal, in the bitstream,a syntax element indicating whether the index block is signaled in thebitstream. In some examples, the video encoder may signal the syntaxelement in a SPS, a PPS, or a slice header in the bitstream. Moreover,in some examples, the video encoder may signal the syntax element on aper CTU basis or a per CU basis. In some such examples, the videodecoder may obtain, from the bitstream, a syntax element indicatingwhether the index block is signaled in the bitstream. The flag can besignaled in a SPS, PPS, a slice header, or another syntax structure, orper CTU or per CU or in any other block sizes. Thus, in some examples,the video decoder may obtain the syntax element from a SPS, a PPS, or aslice header in the bitstream. Furthermore, in some examples, the videodecoder may obtain the syntax element on a per CTU basis or a per CUbasis.

An example of bit plane coding of indexes is now described. In normalline mode, if an index cannot be predicted from top or left; or inhorizontal mode, the line is copied from the leftmost index from thecurrent line, the index value has to be coded directly. In this case, anindex value can be coded bin by bin according to a binary representationof the index value. For example, assuming an index located in line i,column j is denoted by:

C _(i,j) =b _(0ij)+2b _(1ij)+ . . . +2^(N) b _(Nij) =[b _(0ij) b _(1ij). . . b _(Nij)]₂

where b_(kij)=0 or 1. Then b_(kij) can be coded using coded neighboringindex values of b_(kij) as the CABAC contexts. For example, b_(kij) canuse b_(k(i-1)j)+b_(ki(j-1)) as context. b_(kij) may also be coded usingsingle context, or without any context, i.e. bypass coding.

To enable higher throughput, some of the bins of the index are coded inbypass and others using CABAC contexts. For instance, only the MostSignificant Bin of the representation uses context, while the others arecoded in bypass mode.

An example of a flag to indicate ‘other index’ is now described. In thisexample, a one bit flag can be used to indicate whether an index is‘other index’ or not. This flag can be coded using CABAC withsurrounding coded neighbor indexes of the flag as context.

An example of quantization of escape value using a function for rightshift is now described. The table to map from each QP to the amount ofright-shift requires 52 entries. A mapping function may save this memoryrequirements and provide an efficient way to compute the right-shift.For instance, a linear function might be applied:

Right_shift=a*((QP+b)>>c)+d

where a, b, c, and d are integer parameters. A specific example of thisfunction is the following:

Right_shift=(QP>>3)

An example of binarization and coding of indexes is now described. Inthis example, first, a flag is coded using the neighbor coded indexes ascontext to indicate whether the index is zero or not. If the index isnot zero, assume that the index is C>0. Then C-1 is binarized and codedusing bypass CABAC coding. Examples of binarization methods include butare not restricted to: unary, truncated unary, exponential Golomb, orGolomb-Rice with fixed or adaptive parameters.

An example technique for bypass of the indication flag of escape pixelsis now described. In one example, a flag can be used to indicate whethera pixel is an ‘escape pixel’ (i.e. not presented in a major color table)or not. This flag can be bypassed if the number of major colors is lessthan a maximum number of major colors, which implicitly indicates thatno ‘escape pixel’ exists. This maximum number of major colors can bepredefined or adaptively adjusted. When the flag is bypassed, dataindicating the flag is not included in a bitstream.

For instance, in some examples, a video encoder may omit, from abitstream, data indicating the flags if the number of major colors for ablock is less than a maximum allowed number of major colors. Hence, ifthe number of distinct sample values for pixels in the block is lessthan the maximum allowed number of major colors, there can be an entryin the major color table for each of the distinct sample values of thepixels of the block and none of the pixels of the block is an escapepixel. Conversely, if the number of distinct sample values for pixel inthe block is greater than the maximum allowed number of major colors,one or more of the pixels of the block is an escaped pixel. Hence, ifthe number of distinct sample values for pixel in the block is greaterthan the maximum allowed number of major colors, the video encoder maysignal flags to indicate which of the pixels of the block are escapepixels.

In one example, a video decoder may obtain, from a bitstream thatcomprises an encoded representation of the video data, a syntax elementthat indicates whether a pixel in a block is an escape pixel when thenumber of distinct sample values of pixels in the block is greater thana maximum allowed number of colors in a major color table. In thisexample, the video decoder does not obtain the syntax element from thebitstream when the number of distinct sample values of pixels in theblock is less than the maximum allowed number of colors in the majorcolor table. When the pixel is not an escaped pixel, the video decodermay determine, based on an index for the pixel, an entry in the majorcolor table that specifies a sample value for the pixel.

In a similar example, if the number of distinct sample values of pixelsin a block is greater than a maximum allowed number of colors in a majorcolor table, a video encoder may include, in a bitstream that comprisesan encoded representation of the video data, data indicating a syntaxelement indicating whether a pixel of the block is an escaped pixel. Ifthe number of distinct sample values of pixels in a block is less than amaximum allowed number of colors in a major color table, the videoencoder may omit the syntax element from the bitstream. When the pixelis not an escaped pixel, the video encoder may include, in thebitstream, data indicating an index that specifies an entry in the majorcolor table that specifies a sample value for the pixel.

In another example, a flag (or other type of syntax element) can besignaled in the coded bitstream to indicate whether this feature ofbypassing the indication flag of escape pixels is used or not. Forinstance, a video encoder may signal, in the coded bitstream, a syntaxelement to indicate whether or not bypassing an indication syntaxelement (e.g., the indication flag) of escape pixels is used.Accordingly, a video decoder may obtain, from a bitstream, the syntaxelement that indicates whether or not bypassing the indication syntaxelement of escape pixels is used. The flag can be signaled in a SPS,PPS, a slice header, or another structure, or per CTU or per CU or inany other block sizes.

Thus, in some examples, a video encoder may signal, in a bitstream, asecond syntax element indicating whether the bitstream includes a firstsyntax element (i.e., a syntax element indicating whether a pixel is anescape pixel). Furthermore, in some examples, the video decoder mayobtain, from the bitstream, a second syntax element indicating whetherthe bitstream includes a first syntax element (i.e., a syntax elementindicating whether a pixel is an escape pixel). In some examples, thissecond syntax element may be signaled in a sequence parameter set, apicture parameter set, or a slice header. In some examples, the secondsyntax element is signaled on a per CTU basis or a per CU basis.

Example entropy coding methods of quantized escape pixel values orquantized escape prediction errors is now described. In some examples,the quantized escape pixel values (prediction errors) are binarizedusing fixed length codeword. For the first bin of the codeword, CABACcoding is applied with a context modeling. For the remaining bins of thecodeword, CABAC bypass coding is applied with equal probabilities. Inthis example, the length of the codeword is dependent on QP values foreach luminance-chrominance channel (YUV or RGB). For instance, given aninput 8-bit depth data, after quantization of step size 4, the quantizedvalue is in the range of [0, 63], and thus a 6-bit fixed length codewordmay be used instead of an 8-bit codeword, in order to reduce the bits tobe transmitted.

For instance, a video decoder may determine whether a pixel of a pictureof the video data is an escaped pixel. Responsive to determining thatthe pixel is not an escaped pixel, the video decoder may determine anindex for the pixel and determine, based on the index for the pixel, apalette entry that specifies a sample value for the pixel. The paletteentry may be in a palette that comprises palette entries specifyingsample values. Responsive to determining that the pixel is an escapedpixel, the video decoder may use CABAC with context modeling to entropydecode the first bin of a fixed-length codeword. Furthermore, responsiveto determining that the pixel is an escaped pixel, the video decoder mayuse CABAC bypass coding to entropy decode each bin of the fixed-lengthcodeword that follows the first bin of the fixed-length codeword.Furthermore, responsive to determining that the pixel is an escapedpixel, the video decoder may de-binarize the fixed-length codeword todetermine the sample value for the pixel. In some examples, the lengthof the fixed-length codeword is dependent on quantization parameter (QP)values for each channel (e.g., luminance, chrominance, etc. channel) ofthe picture.

In a similar example, a video encoder may determine whether a pixel of apicture of the video data is an escaped pixel. The pixel may be anescaped pixel when a sample value of the pixel corresponds to a samplevalue specified by a palette entry in a palette that comprises paletteentries specifying sample values. Responsive to determining that thepixel is not an escaped pixel, the video encoder may determine an indexfor the pixel and include, in a bitstream that comprises an encodedrepresentation of the video data, data that indicate the index for thepixel. Responsive to determining that the pixel is an escaped pixel, thevideo encoder may binarize the sample value of the pixel to generate afixed-length codeword. Furthermore, responsive to determining that thepixel is an escaped pixel, the video encoder may use CABAC with contextmodeling to entropy encode the first bin of the fixed-length codeword.In addition, responsive to determining that the pixel in an escapedpixel, the video encoder may use CABAC bypass coding to entropy encodeeach bin of the fixed-length codeword that follows the first bin of thefixed-length codeword. In some examples, the length of the fixed-lengthcodeword is dependent on QP values for each channel (e.g., luminance,chrominance, etc. channel) of the picture.

An example technique of palette error limit derivation is now described.In some examples, the palette size is related with QP. For instance, alarger palette error limit may be assigned for larger QP, thus smallergroups of palette indices; smaller palette error limit may be assignedfor smaller QP, thus more groups of palette indices. A mapping table(look-up table) of 52 entries may be used in the memory to store therelationship between each QP value and palette error limit.

For instance, in some examples, a video decoder may determine an indexfor a pixel of a picture of the video data. Furthermore, the videodecoder may determine, based on the index for the pixel, an entry in apalette, wherein the determined entry specifies a sample value for thepixel, wherein a size of the palette is related to a QP. Similarly, insome examples, a video encoder may include, in a bitstream thatcomprises an encoded representation of the video data, data indicatingan index of an entry in a palette, wherein a size of the palette isrelated to a quantization parameter. In some such examples, the videoencoder may determine, based on the quantization parameter, a paletteerror limit. In such examples, the video encoder may include, in thebitstream, the data indicating the index of the entry in the paletteonly if a difference between a sample value specified by the entry and asample value of the pixel is less than the palette error limit.

An example technique of quantized escape pixel derivation is nowdescribed. In some examples, the quantization parameter for an escapepixel (or prediction error) of each channel is the same as thequantization parameter for traditional coefficients coding. In otherwords, the escape pixel (prediction error) quantization ordequantization may be different in different channels. At a videoencoder, each channel of the escape pixel uses the quantizationparameter for traditional coefficient coding. At a video decoder, eachchannel of the escape pixel uses the received quantization parameter fortraditional coefficient coding to reconstruct the escape pixel value orescape pixel prediction error.

In some examples, a video decoder may determine whether a pixel of apicture of the video data is an escaped pixel. Responsive to determiningthat the pixel is not an escaped pixel, the video decoder may determine,based on an index for the pixel, an entry in a palette that comprisesentries specifying sample values, the determined entry specifying asample value of the pixel. Responsive to determining that the pixel isan escaped pixel, the video decoder may determining, based on one ormore syntax elements in the bitstream and without determining an entryin the pallet, the sample value of the pixel. Furthermore, responsive todetermining that the pixel is an escaped pixel, the video decoder mayde-quantize the sample value of the pixel. The sample value of the pixelmay be a residual sample value of the pixel, and the video decoder mayadd a predictive sample value for the pixel to the residual sample valueof the pixel to determine a decoded sample value for the pixel.Furthermore, in some such examples, the sample value of the pixel is afirst sample value of the pixel, the quantization parameter is a firstquantization parameter, the first sample value of the pixel and thefirst quantization parameter correspond to a first channel. In suchexamples, responsive to determining that the pixel is an escaped pixel,the video decode may de-quantize, based on a second quantizationparameter, a second sample value of the pixel, the second sample valueof the pixel and the second quantization parameter corresponding to thesecond channel.

In a similar example, a video encoder may determine whether a pixel of apicture of the video data is an escaped pixel. The pixel may be anescaped pixel when a sample value of the pixel does not correspond to anentry in a palette that comprises entries specifying sample values.Responsive to determining that the pixel is an escaped pixel, the videoencoder may quantize, based on a quantization parameter, the samplevalue of the pixel and include, in a bitstream that comprises an encodedrepresentation of the video data, data indicating the quantized samplevalue of the pixel. Responsive to determining that the pixel is not anescaped pixel, the video encoder may determine an entry in the palettethat corresponds to the sample value of the pixel and include, in thebitstream, data indicating an index to the determined entry in thepalette. In some examples, the sample value for the pixel is a residualsample value of the pixel. Furthermore, in some examples, the samplevalue of the pixel is a first sample value of the pixel, thequantization parameter is a first quantization parameter, the firstsample value of the pixel and the first quantization parametercorrespond to a first channel, and responsive to determining that thepixel is an escaped pixel, the video encoder may quantize, based on asecond quantization parameter, a second sample value of the pixel, thesecond sample value of the pixel and the second quantization parametercorresponding to the second channel. The video encoder may include, inthe bitstream, data indicating the quantized second sample value of thepixel.

FIG. 2 is a block diagram illustrating an example video encoder 20 thatmay implement various techniques of this disclosure. FIG. 2 is providedfor purposes of explanation and should not be considered limiting of thetechniques as broadly exemplified and described in this disclosure. Forpurposes of explanation, this disclosure describes video encoder 20 inthe context of HEVC coding. However, the techniques of this disclosuremay be applicable to other coding standards or methods.

In the example of FIG. 2, video encoder 20 includes a video data memory98, a prediction processing unit 100, a residual generation unit 102, atransform processing unit 104, a quantization unit 106, an inversequantization unit 108, an inverse transform processing unit 110, areconstruction unit 112, a filter unit 114, a decoded picture buffer116, and an entropy encoding unit 118. Prediction processing unit 100includes an inter-prediction processing unit 120 and an intra-predictionprocessing unit 126. Inter-prediction processing unit 120 includes amotion estimation unit and a motion compensation unit (not shown). Videoencoder 20 also includes a palette-based encoding unit 122 configured toperform various aspects of the palette-based coding techniques describedin this disclosure. In other examples, video encoder 20 may includemore, fewer, or different functional components.

Video data memory 98 may store video data to be encoded by thecomponents of video encoder 20. The video data stored in video datamemory 98 may be obtained, for example, from video source 18. Decodedpicture buffer 116 may be a reference picture memory that storesreference video data for use in encoding video data by video encoder 20,e.g., in intra- or inter-coding modes. Video data memory 98 and decodedpicture buffer 116 may be formed by any of a variety of memory devices,such as dynamic random access memory (DRAM), including synchronous DRAM(SDRAM), magnetoresistive RAM (MRAM), resistive RAM (RRAM), or othertypes of memory devices. Video data memory 98 and decoded picture buffer116 may be provided by the same memory device or separate memorydevices. In various examples, video data memory 98 may be on-chip withother components of video encoder 20, or off-chip relative to thosecomponents.

Video encoder 20 may receive video data. Video encoder 20 may encodeeach CTU in a slice of a picture of the video data. Each of the CTUs maybe associated with equally-sized luma coding tree blocks (CTBs) andcorresponding CTBs of the picture. As part of encoding a CTU, predictionprocessing unit 100 may perform quad-tree partitioning to divide theCTBs of the CTU into progressively-smaller blocks. The smaller block maybe coding blocks of CUs. For example, prediction processing unit 100 maypartition a CTB associated with a CTU into four equally-sizedsub-blocks, partition one or more of the sub-blocks into fourequally-sized sub-sub-blocks, and so on.

Video encoder 20 may encode CUs of a CTU to generate encodedrepresentations of the CUs (i.e., coded CUs). As part of encoding a CU,prediction processing unit 100 may partition the coding blocksassociated with the CU among one or more PUs of the CU. Thus, each PUmay be associated with a luma prediction block and corresponding chromaprediction blocks. Video encoder 20 and video decoder 30 may support PUshaving various sizes. As indicated above, the size of a CU may refer tothe size of the luma coding block of the CU and the size of a PU mayrefer to the size of a luma prediction block of the PU. Assuming thatthe size of a particular CU is 2Nx2N, video encoder 20 and video decoder30 may support PU sizes of 2N×2N or N×N for intra prediction, andsymmetric PU sizes of 2N×2N, 2N×N, N×2N, N×N, or similar for interprediction. Video encoder 20 and video decoder 30 may also supportasymmetric partitioning for PU sizes of 2NxnU, 2NxnD, nLx2N, and nRx2Nfor inter prediction.

Inter-prediction processing unit 120 may generate predictive data for aPU by performing inter prediction on each PU of a CU. The predictivedata for the PU may include one or more predictive sample blocks of thePU and motion information for the PU. Inter-prediction unit 121 mayperform different operations for a PU of a CU depending on whether thePU is in an I slice, a P slice, or a B slice. In an I slice, all PUs areintra predicted. Hence, if the PU is in an I slice, inter-predictionunit 121 does not perform inter prediction on the PU. Thus, for blocksencoded in I-mode, the predictive block is formed using spatialprediction from previously-encoded neighboring blocks within the sameframe.

If a PU is in a P slice, the motion estimation unit of inter-predictionprocessing unit 120 may search the reference pictures in a list ofreference pictures (e.g., “RefPicList0”) for a reference region for thePU. The reference region for the PU may be a region, within a referencepicture, that contains sample blocks that most closely correspond to thesample blocks of the PU. The motion estimation unit may generate areference index that indicates a position in RefPicList0 of thereference picture containing the reference region for the PU. Inaddition, the motion estimation unit may generate an MV that indicates aspatial displacement between a coding block of the PU and a referencelocation associated with the reference region. For instance, the MV maybe a two-dimensional vector that provides an offset from the coordinatesin the current decoded picture to coordinates in a reference picture.The motion estimation unit may output the reference index and the MV asthe motion information of the PU. The motion compensation unit ofinter-prediction processing unit 120 may generate the predictive sampleblocks of the PU based on actual or interpolated samples at thereference location indicated by the motion vector of the PU.

If a PU is in a B slice, the motion estimation unit may performuni-prediction or bi-prediction for the PU. To perform uni-predictionfor the PU, the motion estimation unit may search the reference picturesof RefPicList0 or a second reference picture list (“RefPicList1”) for areference region for the PU. The motion estimation unit may output, asthe motion information of the PU, a reference index that indicates aposition in RefPicList0 or RefPicList1 of the reference picture thatcontains the reference region, an MV that indicates a spatialdisplacement between a sample block of the PU and a reference locationassociated with the reference region, and one or more predictiondirection indicators that indicate whether the reference picture is inRefPicList0 or RefPicList1. The motion compensation unit ofinter-prediction processing unit 120 may generate the predictive sampleblocks of the PU based at least in part on actual or interpolatedsamples at the reference region indicated by the motion vector of thePU.

To perform bi-directional inter prediction for a PU, the motionestimation unit may search the reference pictures in RefPicList0 for areference region for the PU and may also search the reference picturesin RefPicList1 for another reference region for the PU. The motionestimation unit may generate reference picture indexes that indicatepositions in RefPicList0 and RefPicList1 of the reference pictures thatcontain the reference regions. In addition, the motion estimation unitmay generate MVs that indicate spatial displacements between thereference location associated with the reference regions and a sampleblock of the PU. The motion information of the PU may include thereference indexes and the MVs of the PU. The motion compensation unitmay generate the predictive sample blocks of the PU based at least inpart on actual or interpolated samples at the reference region indicatedby the motion vector of the PU.

In accordance with various examples of this disclosure, video encoder 20may be configured to perform palette-based coding. With respect to theHEVC framework, as an example, the palette-based coding techniques maybe configured to be used as a CU mode. In other examples, thepalette-based coding techniques may be configured to be used as a PUmode in the framework of HEVC. Accordingly, all of the disclosedprocesses described herein (throughout this disclosure) in the contextof a CU mode may, additionally or alternatively, apply to a PU mode.However, these HEVC-based examples should not be considered arestriction or limitation of the palette-based coding techniquesdescribed herein, as such techniques may be applied to workindependently or as part of other existing or yet to be developedsystems/standards. In these cases, the unit for palette coding can besquare blocks, rectangular blocks or even regions of non-rectangularshape.

Palette-based encoding unit 122, for example, may perform palette-baseddecoding when a palette-based encoding mode is selected, e.g., for a CUor PU. For example, palette-based encoding unit 122 may be configured togenerate a palette having entries indicating pixel values, select pixelvalues in a palette to represent pixel values of at least some positionsof a block of video data, and signal information associating at leastsome of the positions of the block of video data with entries in thepalette corresponding, respectively, to the selected pixel values.Although various functions are described as being performed bypalette-based encoding unit 122, some or all of such functions may beperformed by other processing units, or a combination of differentprocessing units.

Palette-based encoding unit 122 may be configured to generate any of thevarious syntax elements described herein. Accordingly, video encoder 20may be configured to encode blocks of video data using palette-basedcode modes as described in this disclosure. Video encoder 20 mayselectively encode a block of video data using a palette coding mode, orencode a block of video data using a different mode, e.g., such an HEVCinter-predictive or intra-predictive coding mode. The block of videodata may be, for example, a CU or PU generated according to an HEVCcoding process. A video encoder 20 may encode some blocks withinter-predictive temporal prediction or intra-predictive spatial codingmodes and decode other blocks with the palette-based coding mode.

Intra-prediction processing unit 126 may generate predictive data for aPU by performing intra prediction on the PU. The predictive data for thePU may include predictive sample blocks for the PU and various syntaxelements. Intra-prediction processing unit 126 may perform intraprediction on PUs in I slices, P slices, and B slices.

To perform intra prediction on a PU, intra-prediction processing unit126 may use multiple intra prediction modes to generate multiple sets ofpredictive data for the PU. When using some intra prediction modes togenerate a set of predictive data for the PU, intra-predictionprocessing unit 126 may extend values of samples from sample blocks ofneighboring PUs across the predictive blocks of the PU in directionsassociated with the intra prediction modes. The neighboring PUs may beabove, above and to the right, above and to the left, or to the left ofthe PU, assuming a left-to-right, top-to-bottom encoding order for PUs,CUs, and CTUs. Intra-prediction processing unit 126 may use variousnumbers of intra prediction modes, e.g., 33 directional intra predictionmodes. In some examples, the number of intra prediction modes may dependon the size of the region associated with the PU.

Prediction processing unit 100 may select the predictive data for PUs ofa CU from among the predictive data generated by inter-predictionprocessing unit 120 for the PUs or the predictive data generated byintra-prediction processing unit 126 for the PUs. In some examples,prediction processing unit 100 selects the predictive data for the PUsof the CU based on rate/distortion metrics of the sets of predictivedata. The predictive sample blocks of the selected predictive data maybe referred to herein as the selected predictive sample blocks.

Residual generation unit 102 may generate, based on the coding blocks(e.g., luma, Cb and Cr coding blocks) of a CU and the selectedpredictive sample blocks (e.g., predictive luma, Cb and Cr blocks) ofthe PUs of the CU, residual blocks (e.g., luma, Cb and Cr residualblocks) of the CU. For instance, residual generation unit 102 maygenerate the residual blocks of the CU such that each sample in theresidual blocks has a value equal to a difference between a sample in acoding block of the CU and a corresponding sample in a correspondingselected predictive sample block of a PU of the CU.

Transform processing unit 104 may perform quad-tree partitioning topartition the residual blocks associated with a CU into transform blocksassociated with TUs of the CU. Thus, in some examples, a TU may beassociated with a luma transform block and two chroma transform blocks.The sizes and positions of the luma and chroma transform blocks of TUsof a CU may or may not be based on the sizes and positions of predictionblocks of the PUs of the CU. A quad-tree structure known as a “residualquad-tree” (RQT) may include nodes associated with each of the regions.The TUs of a CU may correspond to leaf nodes of the RQT.

Transform processing unit 104 may generate transform coefficient blocksfor each TU of a CU by applying one or more transforms to the transformblocks of the TU. Transform processing unit 104 may apply varioustransforms to a transform block associated with a TU. For example,transform processing unit 104 may apply a discrete cosine transform(DCT), a directional transform, or a conceptually similar transform to atransform block. In some examples, transform processing unit 104 doesnot apply transforms to a transform block. In such examples, thetransform block may be treated as a transform coefficient block.

Quantization unit 106 may quantize the transform coefficients in acoefficient block. The quantization process may reduce the bit depthassociated with some or all of the transform coefficients. For example,an n-bit transform coefficient may be rounded down to an m-bit transformcoefficient during quantization, where n is greater than m. Quantizationunit 106 may quantize a coefficient block associated with a TU of a CUbased on a quantization parameter (QP) value associated with the CU.Video encoder 20 may adjust the degree of quantization applied to thecoefficient blocks associated with a CU by adjusting the QP valueassociated with the CU. Quantization may introduce loss of information,thus quantized transform coefficients may have lower precision than theoriginal ones.

Inverse quantization unit 108 and inverse transform processing unit 110may apply inverse quantization and inverse transforms to a coefficientblock, respectively, to reconstruct a residual block from thecoefficient block. Reconstruction unit 112 may add the reconstructedresidual block to corresponding samples from one or more predictivesample blocks generated by prediction processing unit 100 to produce areconstructed transform block associated with a TU. By reconstructingtransform blocks for each TU of a CU in this way, video encoder 20 mayreconstruct the coding blocks of the CU.

Filter unit 114 may perform one or more deblocking operations to reduceblocking artifacts in the coding blocks associated with a CU. Decodedpicture buffer 116 may store the reconstructed coding blocks afterfilter unit 114 performs the one or more deblocking operations on thereconstructed coding blocks. Inter-prediction processing unit 120 mayuse a reference picture that contains the reconstructed coding blocks toperform inter prediction on PUs of other pictures. In addition,intra-prediction processing unit 126 may use reconstructed coding blocksin decoded picture buffer 116 to perform intra prediction on other PUsin the same picture as the CU.

Entropy encoding unit 118 may receive data from other functionalcomponents of video encoder 20. For example, entropy encoding unit 118may receive coefficient blocks from quantization unit 106 and mayreceive syntax elements from prediction processing unit 100. Entropyencoding unit 118 may perform one or more entropy encoding operations onthe data to generate entropy-encoded data. For example, entropy encodingunit 118 may perform a CABAC operation, a context-adaptive variablelength coding (CAVLC) operation, a variable-to-variable (V2V) lengthcoding operation, a syntax-based context-adaptive binary arithmeticcoding (SBAC) operation, a Probability Interval Partitioning Entropy(PIPE) coding operation, an Exponential-Golomb encoding operation, oranother type of entropy encoding operation on the data. Video encoder 20may output a bitstream that includes entropy-encoded data generated byentropy encoding unit 118. For instance, the bitstream may include datathat represents a RQT for a CU.

In some examples, residual coding is not performed with palette coding.Accordingly, video encoder 20 may not perform transformation orquantization when coding using a palette coding mode. In addition, videoencoder 20 may entropy encode data generated using a palette coding modeseparately from residual data.

According to one or more of the techniques of this disclosure, videoencoder 20, and specifically palette-based encoding unit 122, mayperform palette-based video coding of predicted video blocks. Asdescribed above, a palette generated by video encoder 20 may beexplicitly encoded and sent to video decoder 30, predicted from previouspalette entries, predicted from previous pixel values, or a combinationthereof.

Palette-based encoding unit 122 may apply techniques of this disclosureto bypass encoding of a map of palette index values for pixels of apalette-coded block under certain circumstances. According to aspects ofthis disclosure, palette-based encoding unit 122 may bypass encoding ofthe map of palette index values for a palette-coded block ifpalette-based encoding unit 122 determines that all pixels of the blockare of the same color. For instance, palette-based encoding unit 122 maydetermine that a palette-coded CU of a picture is a “single color CU”and may bypass encoding and signaling of the map of palette index valuesfor the single color CU.

More specifically, palette-based encoding unit 122 may determine whethera palette-coded CU is a single color CU if the current CU satisfies twoconditions. The first condition that palette-based encoding unit 122 mayuse in determining whether the palette-coded CU is a single color CU iswhether the size of the corresponding palette is equal to one (1). Ifthe size of the palette is equal to one, then palette-based encodingunit 122 may determine that the first condition is met with respect tothe palette-coded CU being a single color CU. More specifically, if thepalette size is one, then palette-based encoding unit 122 may determinethat the palette includes only one color that corresponds to(non-escape) pixels of the palette-coded CU. In some examples where thepalette size is one, palette-based encoding unit 122 may determine thatthe only index value included in the palette is zero (0).

If palette-based encoding unit 122 determines that the palette size forthe palette-coded CU is 1 (i.e., that the first condition is met), thenpalette-based encoding unit 122 may determine whether the palette-codedCU meets a second condition to be a single color CU. The secondcondition that palette-based encoding unit 122 may use in determiningwhether the palette-coded CU is a single color CU is that thepalette-coded CU does not include any escape pixels. If thepalette-coded CU includes at least one escape pixel, then video encoder20 may determine that, even though the corresponding palette indicatesonly one color with respect to the palette-coded CU, the palette-codedCU includes pixels of two or more colors. For instance, thepalette-coded CU may include at least one pixel that has the colorindicated in the palette, and at least one escape pixel that has adifferent color.

If palette-based encoding unit 122 determines that the palette-coded CUsatisfies both of the conditions described above, then palette-basedencoding unit 122 may determine that the palette-coded CU is a singlecolor CU. More specifically, if the palette-coded CU is associated witha single-entry palette (shown by the palette size of one), and thepalette-coded CU does not include any escape pixels, then palette-basedencoding unit 122 may determine that all individual pixels of thepalette-coded CU are of the same color (i.e., the color indicated by thesingle entry of the corresponding palette). In various implementations,palette-based encoding unit 122 may apply the single color CUidentification techniques described above with respect to a single colorcomponent basis with respect to the palette, or to a combined index thatindicates more than one color component.

Responsive to determining that the palette-coded CU includes pixels ofonly one color (i.e., that the CU is a single color CU), palette-basedencoding unit 122 may implement techniques of this disclosure to bypassencoding the map of palette index values for the pixels of the singlecolor CU. By bypassing the encoding of the map of palette index valuesfor the CU, palette-based encoding unit 122 may conserve computingresources and bandwidth that would otherwise be expended for encodingand signaling color information for the CU. Instead of encoding a map ofpalette index values for the pixels of the single color CU,palette-based encoding unit 122 may implement techniques of thisdisclosure to more efficiently indicate the color information for theentire set of pixels that make up the single color CU. In cases wherepalette-based encoding unit 122 bypasses encoding of the palette indexfor all individual pixels of a detected single color CU, video encoder20 (or one or more components thereof) may not need to signal a map ofpalette index values for the pixels of the CU, thereby conservingcomputing resources and reducing bandwidth consumption.

According to some examples of this disclosure, palette-based encodingunit 122 may encode a flag to indicate that video encoder 20 bypassedencoding (and signaling) of the map of palette index values on aline-by-line basis for the single color CU. By encoding a one-bit flagfor the entire CU instead of the map (of any size) of palette indexvalues for the pixels of the CU, palette-based encoding unit 122 maycause video encoder 20 to conserve computing resources and signalingbandwidth in comparison to existing palette-based coding techniques.Moreover, palette-based encoding unit 122 may maintain precision andquality of the encoded single color CU, because the single-entry paletteencoded by palette-based encoding unit 122 and signaled by video encoder20 for the CU includes the color information for all pixels of the CU.In various examples, palette-based encoding unit 122 may encode the flagin various ways, such as in an SPS, a PPS, or a slice header. In variousexamples, palette-based encoding unit 122 may encode the flag on aper-CTU basis, a per-CU basis, or for a block of any block size, aswell.

In other examples, palette-based encoding unit 122 may implementtechniques of this disclosure to derive an error limit for a palette fora palette-coded block. As used herein, the terms “error limit” or“palette error limit” may refer to the minimum variation of pixel values(e.g., in terms of color information) that the entries of the palettecan include. More specifically, the palette error limit defines aminimum variation in color value that any two palette entries mustdisplay. As described, to encode a block according to palette-basedcoding, palette-based encoding unit 122 may construct the correspondingpalette to include color values that occur most frequently (on apixel-by-pixel basis) within the block.

In other words, palette-based encoding unit 122 may construct a paletteby clustering high-occurrence (or relatively higher-occurrence) pixelvalues of the block in the palette. Palette-based encoding unit 122 mayselect the high-occurrence pixel values such that the pixel valuesdisplay at least a particular minimum variation, in terms of colorinformation. In turn, the minimum variation of pixel value within theselected set of high-occurrence pixel values may form the error limit ofthe corresponding palette. It will be appreciated that, while thepalette error limit may include several pixel values, the palette maynot necessarily include every pair of pixel values that display at leastthe palette error limit. Thus, the same palette error limit may apply topalettes of varying sizes. Palette-based encoding unit 122 may use thepalette error limit in making determinations as to the color values thatare to be included in the palette.

Palette-based encoding unit 122 may implement techniques of thisdisclosure to define the error limit for a palette. According to variousaspects of this disclosure, palette-based encoding unit 122 maydetermine the palette error limit based on the quantization parameter(QP) for the palette-coded block. In various examples, palette-basedencoding unit 122 may determine that the palette error limit is directlyproportional to the QP value for the corresponding block. Morespecifically, in these examples, palette-based encoding unit may assigna larger error limit for a palette for a block that is quantized with agreater QP value, and a smaller error limit for a palette for a blockthat is quantized with a lesser QP value.

Additionally, palette-based encoding unit 122 may implement thetechniques described herein to derive the palette error limit by storinga table that maps the QP value of a palette-coded block to thecorresponding palette error limit. In turn, palette-based encoding unit122 may access a particular palette error limit entry from the table inorder to determine the corresponding palette. By using a table that mapsthe palette error limit to the QP of a palette-coded block,palette-based encoding unit 122 may improve computational efficiency incomparison to existing palette-based coding techniques. Morespecifically, by implementing the mapping table-based palette errorlimit derivation techniques described herein, palette-based encodingunit 122 may conserve resources that would otherwise be expendedaccording to existing techniques, which rely on executing a function toderive the error limit for each palette.

Thus, palette-based encoding unit 122 may define palettes that canaccommodate a greater number of palette indices for require a greatervariation between pixel values for blocks that are quantized withgreater QP values, and may define palettes that require a lesservariation between pixel values that are quantized with greater QPvalues. In this manner, palette-based encoding unit 122 may implementvarious techniques of this disclosure to customize the palette (based onthe palette's error limit) based on the QP value with which thecorresponding block is quantized, thereby determining the contents ofthe palette for a block based on the block's QP value. Additionally,palette-based encoding unit 122 may generate and/or store a table (e.g.,a mapping table or look-up table) to reflect the relationship betweeneach QP value and the corresponding palette error limit. In this manner,palette-based encoding unit 122 may implement the techniques of thisdisclosure to derive a palette error limit for a corresponding palettein a less computationally expensive way than in existing palette-codingtechniques, which rely on solving a function to derive the error limitfor each palette.

Palette-based encoding unit 122 may, in some examples, implement varioustechniques of this disclosure for quantized escape pixel derivation.More specifically, palette-based encoding unit 122 may implement thetechniques to define the quantization value of the QP for an escapepixel. For example, according to palette-based coding techniques, ifpalette-based encoding unit 122 detects an escape pixel in apalette-coded block, palette-based encoding unit 122 may encode thepixel value, or a prediction error thereof, because the correspondingpalette does not include any entries for the escape pixel. Additionally,to conserve signaling bandwidth, quantization unit 106 may quantize theencoded pixel value of the escape pixel prior to signaling by othercomponents of video encoder 20.

According to existing palette-based coding techniques, no quantizationvalue (QP value) was defined for quantizing an escape pixel.Palette-based encoding unit 122 may implement techniques of thisdisclosure to define the QP value for quantizing an escape pixel. Morespecifically, palette-based encoding unit 122 may define the QP valuefor an escape pixel as equal to the QP value for traditional coefficientencoding within the same color channel. Thus, palette-based encodingunit 122 may quantize all escape pixels according to a single QP value,within a given color channel. Additionally, as palette-based encodingunit 122 may define the QP value for all escape pixels only within asingle color channel, palette-based encoding unit 122 may use differentQP values for quantizing escape pixels with respect to different colorchannels.

Additionally, palette-based encoding unit 122 may implement othertechniques of this disclosure to use a flag to indicate and/or detectthe inclusion of an escape pixel in a palette-coded block. According toexisting palette-based coding techniques, escape pixels may be signaledand detected using a “reserved” palette index value. For instance,according to the existing palette-based coding techniques, the reservedpalette index value that indicates an escape pixel may be 32. Morespecifically, the palette index value of 32 may be used for all escapepixels, regardless of whether two escape pixels have different pixelvalues. Thus, according to the existing palette-based coding techniques,video coding devices may use a five-bit value (of 32) for each escapepixel of a palette-coded block.

Palette-based encoding unit 122 may implement techniques of thisdisclosure to conserve computing resources (e.g., storage and memory)and reduce bandwidth consumption, while maintaining picture precisionwith respect to signaling an indication of an escape pixel in apalette-coded block. For instance, palette-based encoding unit 122 mayencode a flag (e.g., a one-bit data unit) to indicate whether a pixel ina palette-coded block is an escape pixel. As described herein, the flag,when enabled, may indicate that the associated pixel is assigned apalette index referred to as “other index.” Palette-based encoding unit122 may use the “other index” status of the flag to replace the paletteindex value of 32 that is traditionally used to indicate an escape pixelwith respect to the palette. Thus, palette-based encoding unit 122 mayencode (and other components of video encoder 20 may signal) a one-bitflag instead of a five-bit index value to indicate that a pixel of apalette-coded block is an escape pixel. In turn, video encoder 20 (orone or more components thereof) may encode and signal the pixel value,or residual data thereof, in the encoded video bitstream.

One or both of palette-based encoding unit 122 and quantization unit 106may also implement techniques of this disclosure to quantize the pixelvalues of escape pixels in accordance with palette-based coding. Forinstance, palette-based encoding unit 122 and/or quantization unit 106may conserve computing resources (e.g., memory usage, processor clockcycles, etc.) by quantizing the pixel values of escape pixels accordingto aspects of this disclosure. In some examples, quantization unit 106may implement the techniques described herein to quantize the escapepixel values by substituting divide operations with shift operations(e.g., right-shift operations). More specifically, quantization unit 106may determine the specific right-shift operation based on the QP valueof the corresponding escape pixel. For instance, quantization unit 106may derive the right-shift value by solving a function that includes theQP value as a parameter or operand.

For instance, according to existing techniques, palette-based encodingunit 122 may form a table that maps the QP value of each escape pixel tothe amount of the right-shift to apply to the pixel value. Palette-basedencoding unit 122 may form the table to include 52 entries. For example,the 52-entry mapping table may provide a right-shift amountcorresponding to each possible QP value for a given escape pixel. Inturn, quantization unit 106 may apply a mapping operation to determinethe right-shift amount for each pixel, based on the corresponding QPvalue entry in the table.

In various examples, quantization unit 106 may implement varioustechniques of this disclosure to quantize an escape pixel by determiningthe right-shift amount for an escape pixel based on applying a linearfunction to the escape pixel value to determine the right-shift amount.An example of a linear function that quantization unit 106 may apply toderive the right-shift amount for quantizing an escape pixel is asfollows:

Right_shift=a*((QP+b)>>c)+d,

where a, b, c, and d are all integer parameters. Additionally, the “>>”operator denotes a right-shift operation. In a specific result ofapplying the equation above, quantization unit 106 may determine thatthe right-shift amount for an escape pixel value is three. The resultingright-shift operation may be expressed as Right_shift=(QP>>3). Bysolving a function (e.g., the linear function described above),quantization unit 106 may enable a decoding device (e.g., video decoder30 and/or various components thereof) to dequantize an escape pixel,while conserving storage resources that would otherwise be expended tostore the 52-entry table. In this manner, quantization unit 106 mayimplement techniques of this disclosure to quantize an escape pixelwhile mitigating storage requirements with respect to video decoder 30.

The techniques described in this disclosure may also include techniquesfor various combinations of one or more of signaling palette-basedcoding modes, transmitting palettes, predicting palettes, derivingpalettes, or transmitting palette-based coding maps and other syntaxelements.

FIG. 3 is a block diagram illustrating an example video decoder 30 thatis configured to implement the techniques of this disclosure. FIG. 3 isprovided for purposes of explanation and is not limiting on thetechniques as broadly exemplified and described in this disclosure. Forpurposes of explanation, this disclosure describes video decoder 30 inthe context of HEVC coding. However, the techniques of this disclosuremay be applicable to other coding standards or methods.

In the example of FIG. 3, video decoder 30 includes a video data memory148, an entropy decoding unit 150, a prediction processing unit 152, aninverse quantization unit 154, an inverse transform processing unit 156,a reconstruction unit 158, a filter unit 160, and a decoded picturebuffer 162. Prediction processing unit 152 includes a motioncompensation unit 164 and an intra-prediction processing unit 166. Videodecoder 30 also includes a palette-based decoding unit 165 configured toperform various aspects of the palette-based coding techniques describedin this disclosure. In other examples, video decoder 30 may includemore, fewer, or different functional components.

Video data memory 148 may store video data, such as an encoded videobitstream, to be decoded by the components of video decoder 30. Thevideo data stored in video data memory 148 may be obtained, for example,from computer-readable medium 16, e.g., from a local video source, suchas a camera, via wired or wireless network communication of video data,or by accessing physical data storage media. Video data memory 148 mayform a coded picture buffer (CPB) that stores encoded video data from anencoded video bitstream. Decoded picture buffer 162 may be a referencepicture memory that stores reference video data for use in decodingvideo data by video decoder 30, e.g., in intra- or inter-coding modes.Video data memory 148 and decoded picture buffer 162 may be formed byany of a variety of memory devices, such as dynamic random access memory(DRAM), including synchronous DRAM (SDRAM), magnetoresistive RAM (MRAM),resistive RAM (RRAM), or other types of memory devices. Video datamemory 148 and decoded picture buffer 162 may be provided by the samememory device or separate memory devices. In various examples, videodata memory 148 may be on-chip with other components of video decoder30, or off-chip relative to those components.

Video data memory 148, i.e., a CPB, may receive and store encoded videodata (e.g., NAL units) of a bitstream. Entropy decoding unit 150 mayreceive encoded video data (e.g., NAL units) from video data memory 148and may parse the NAL units to decode syntax elements. Entropy decodingunit 150 may entropy decode entropy-encoded syntax elements in the NALunits. Prediction processing unit 152, inverse quantization unit 154,inverse transform processing unit 156, reconstruction unit 158, andfilter unit 160 may generate decoded video data based on the syntaxelements obtained (e.g., extracted) from the bitstream.

The NAL units of the bitstream may include coded slice NAL units. Aspart of decoding the bitstream, entropy decoding unit 150 may extractand entropy decode syntax elements from the coded slice NAL units. Eachof the coded slices may include a slice header and slice data. The sliceheader may contain syntax elements pertaining to a slice. The syntaxelements in the slice header may include a syntax element thatidentifies a PPS associated with a picture that contains the slice.

In addition to decoding syntax elements from the bitstream, videodecoder 30 may perform a reconstruction operation on a non-partitionedCU. To perform the reconstruction operation on a non-partitioned CU,video decoder 30 may perform a reconstruction operation on each TU ofthe CU. By performing the reconstruction operation for each TU of theCU, video decoder 30 may reconstruct residual blocks of the CU.

As part of performing a reconstruction operation on a TU of a CU,inverse quantization unit 154 may inverse quantize, i.e., de-quantize,coefficient blocks associated with the TU. Inverse quantization unit 154may use a QP value associated with the CU of the TU to determine adegree of quantization and, likewise, a degree of inverse quantizationfor inverse quantization unit 154 to apply. That is, the compressionratio, i.e., the ratio of the number of bits used to represent originalsequence and the compressed one, may be controlled by adjusting thevalue of the QP used when quantizing transform coefficients. Thecompression ratio may also depend on the method of entropy codingemployed.

After inverse quantization unit 154 inverse quantizes a coefficientblock, inverse transform processing unit 156 may apply one or moreinverse transforms to the coefficient block in order to generate aresidual block associated with the TU. For example, inverse transformprocessing unit 156 may apply an inverse DCT, an inverse integertransform, an inverse Karhunen-Loeve transform (KLT), an inverserotational transform, an inverse directional transform, or anotherinverse transform to the coefficient block.

If a PU is encoded using intra prediction, intra-prediction processingunit 166 may perform intra prediction to generate predictive blocks forthe PU. Intra-prediction processing unit 166 may use an intra predictionmode to generate the predictive luma, Cb and Cr blocks for the PU basedon the prediction blocks of spatially-neighboring PUs. Intra-predictionprocessing unit 166 may determine the intra prediction mode for the PUbased on one or more syntax elements decoded from the bitstream.

Prediction processing unit 152 may construct a first reference picturelist (RefPicList0) and a second reference picture list (RefPicList1)based on syntax elements extracted from the bitstream. Furthermore, if aPU is encoded using inter prediction, entropy decoding unit 150 mayextract motion information for the PU. Motion compensation unit 164 maydetermine, based on the motion information of the PU, one or morereference regions for the PU. Motion compensation unit 164 may generate,based on samples blocks at the one or more reference blocks for the PU,predictive blocks (e.g., predictive luma, Cb and Cr blocks) for the PU.

Reconstruction unit 158 may use the transform blocks (e.g., luma, Cb andCr transform blocks) associated with TUs of a CU and the predictiveblocks (e.g., luma, Cb and Cr blocks) of the PUs of the CU, i.e., eitherintra-prediction data or inter-prediction data, as applicable, toreconstruct the coding blocks (e.g., luma, Cb and Cr coding blocks) ofthe CU. For example, reconstruction unit 158 may add samples of thetransform blocks (e.g., luma, Cb and Cr transform blocks) tocorresponding samples of the predictive blocks (e.g., predictive luma,Cb and Cr blocks) to reconstruct the coding blocks (e.g., luma, Cb andCr coding blocks) of the CU.

Filter unit 160 may perform a deblocking operation to reduce blockingartifacts associated with the coding blocks (e.g., luma, Cb and Crcoding blocks) of the CU. Video decoder 30 may store the coding blocks(e.g., luma, Cb and Cr coding blocks) of the CU in decoded picturebuffer 162. Decoded picture buffer 162 may provide reference picturesfor subsequent motion compensation, intra prediction, and presentationon a display device, such as display device 32 of FIG. 1. For instance,video decoder 30 may perform, based on the blocks (e.g., luma, Cb and Crblocks) in decoded picture buffer 162, intra prediction or interprediction operations on PUs of other CUs. In this way, video decoder 30may extract, from the bitstream, transform coefficient levels of asignificant coefficient block, inverse quantize the transformcoefficient levels, apply a transform to the transform coefficientlevels to generate a transform block, generate, based at least in parton the transform block, a coding block, and output the coding block fordisplay.

In accordance with various examples of this disclosure, video decoder 30may be configured to perform palette-based coding. Palette-baseddecoding unit 165, for example, may perform palette-based decoding whena palette-based decoding mode is selected, e.g., for a CU or PU. Forexample, palette-based decoding unit 165 may be configured to generate apalette having entries indicating pixel values. Furthermore, in thisexample, palette-based decoding unit 165 may receive informationassociating at least some positions of a block of video data withentries in the palette. In this example, palette-based decoding unit 165may select pixel values in the palette based on the information.Additionally, in this example, palette-based decoding unit 165 mayreconstruct pixel values of the block based on the selected pixelvalues. Although various functions are described as being performed bypalette-based decoding unit 165, some or all of such functions may beperformed by other processing units, or a combination of differentprocessing units.

Palette-based decoding unit 165 may receive palette coding modeinformation, and perform the above operations when the palette codingmode information indicates that the palette coding mode applies to theblock. When the palette coding mode information indicates that thepalette coding mode does not apply to the block, or when other modeinformation indicates the use of a different mode, palette-baseddecoding unit 165 decodes the block of video data using a non-palettebased coding mode, e.g., such an HEVC inter-predictive orintra-predictive coding mode, when the palette coding mode informationindicates that the palette coding mode does not apply to the block. Theblock of video data may be, for example, a CU or PU generated accordingto an HEVC coding process. A video decoder 30 may decode some blockswith inter-predictive temporal prediction or intra-predictive spatialcoding modes and decode other blocks with the palette-based coding mode.The palette-based coding mode may comprise one of a plurality ofdifferent palette-based coding modes, or there may be a singlepalette-based coding mode.

According to one or more of the techniques of this disclosure, videodecoder 30, and specifically palette-based decoding unit 165, mayperform palette-based video decoding of palette-coded video blocks. Asdescribed above, a palette decoded by video decoder 30 may be explicitlyencoded and signaled by video encoder 20, reconstructed by video decoder30 with respect to a received palette-coded block, predicted fromprevious palette entries, predicted from previous pixel values, or acombination thereof.

As described above, if a palette-coded video block satisfies aparticular set of conditions, video encoder 20 (and various componentthereof, such as palette-based encoding unit 122) may bypass encodingand signaling of a map of palette index values for the pixels of theblock. In examples where video encoder 20 bypasses encoding andsignaling of the map of palette index values for a single color blockthat is palette-coded, video decoder 30 (and specific componentsthereof, such as palette-based decoding unit 165) may apply varioustechniques of this disclosure to reconstruct the single color block. Insome examples, palette-based decoding unit 165 may perform operationsreciprocal to those described above with respect to palette-basedencoding unit 122 of video encoder 20, to determine that thepalette-coded block is a single color block. For instance, palette-baseddecoding unit 165 may determine that the palette for the current blockhas a size of one, thereby determining that the block satisfies thefirst condition to qualify as a single color block. In various examples,video decoder 30 may receive the palette in an encoded video bitstreamfrom video encoder 20, or may derive the palette using various otherdata included in the encoded video bitstream.

Additionally, video decoder 30 may determine that the block does notinclude any escape pixels, thereby determining that the block satisfiesthe second condition to qualify as a single color block. Based ondetermining that the size of the palette for the block is one (thussatisfying the first condition), and that the block does not include anyescape pixels (thus satisfying the second condition), palette-baseddecoding unit 165 may implement techniques of this disclosure todetermine that the current block is a single color block. In turn,palette-based decoding unit 165 may reconstruct the single color blockby reconstructing all pixels of the single color block according to thecolor information indicated in the single entry of the correspondingpalette. In this manner, palette-based decoding unit 165 may implementtechniques of this disclosure to reconstruct a palette-coded blockaccurately, while conserving computing resources and bandwidth thatwould otherwise be required to reconstruct the block by relying on a mapof palette index values for all pixels of the block.

In some examples, video decoder 30 may receive, in the encoded videobitstream, a flag that indicates whether video encoder 20 bypassedencoding and signaling of the map of palette index values for one ormore pixels of a palette-encoded block, in accordance with techniques ofthis disclosure. In cases where video decoder 30 receives a flagindicating that video encoder 20 did bypass encoding and signaling ofthe map of palette index values for the palette-encoded block,palette-based decoding unit 165 may implement techniques of thisdisclosure to determine that the current block is palette-coded, and isa single color block. More specifically, if the flag is enabled (e.g.,set to a value of one), palette-based decoding unit 165 may determinethat the palette-coded block is a single color block. In turn,palette-based decoding unit 165 may implement techniques of thisdisclosure to reconstruct all pixels of the block according to the colorinformation of the single entry in the palette for the block. Thus,palette-based decoding unit 165 may implement techniques of thisdisclosure to accurately reconstruct the palette-encoded block using aone-bit flag for the entire block, rather than using separate indexvalues (of varying bitdepth) for different pixels, or groups of pixels(e.g. a line), of the block. In this manner, palette-based decoding unit165 may conserve computing resource expenditure at video decoder 30 inreconstructing single color palette-coded blocks, and may reduce thebandwidth required by video decoder 30 to receive the data necessary toreconstruct the single color palette coded blocks, while maintainingprecision and picture quality.

As described, video encoder 20 (and components thereof, such aspalette-based encoding unit 122 and/or quantization unit 106) mayimplement certain techniques of this disclosure to quantize escape pixelvalues of a palette-coded block with enhanced computing efficiency.Video decoder 30 (and various components thereof, such as palette-baseddecoding unit 165 and/or inverse quantization unit 154) may performreciprocal operations to those described above with respect to videoencoder 20, to dequantize escape pixels in accordance with varioustechniques of this disclosure. For instance, inverse quantization unit154 may dequantize all escape pixels of a single color channel using thesame QP value, based on information received in the encoded videobitstream from video encoder 20. More specifically, in accordance withaspects of this disclosure, inverse quantization unit 154 may dequantizeany escape pixels (or prediction errors/residual values thereof)communicated over a particular color channel, using a QP value that isdetermined based on the QP value for traditional transform coefficientdequantization for blocks communicated over the current color channel.In some examples, inverse quantization unit 154 may implement thetechniques of this disclosure to dequantize escape pixels communicatedover different color channels using different QP values, based on the QPvalue used for traditional transform coefficient coding being differentamong the different channels.

In this manner, video decoder 30 may implement the techniques describedherein to define and apply a single QP value (to dequantize) all escapepixels communicated over a particular color channel. Thus, video decoder30 may apply aspects of this disclosure to define a QP value for escapepixels detected through palette-based coding, where existingpalette-based coding techniques did not define a QP value for escapepixels.

In some examples, components of video decoder 30, such as inversequantization unit 154, may implement techniques of this disclosure toperform reciprocal operations of those described above with respect tovideo encoder 20 (and/or components thereof, such as quantization unit106), to dequantize a quantized escape pixel value. For instance,inverse quantization unit 154 may implement techniques of thisdisclosure to calculate a shift amount (e.g., for a correspondingleft-shift operation) based on a QP value in dequantizing thecorresponding quantized escape pixel value. In this manner, inversequantization unit 154 may also apply aspects of this disclosure toconserve computing resources, such as storage utilization, by leveraginga function instead of storing a 52-entry mapping table.

FIG. 4 is a flowchart illustrating an example process 180 by which avideo decoding device may implement techniques of this disclosure tobypass decoding of index values for pixels of a palette-coded block,based on a particular set of conditions. While process 180 may beperformed by a variety of devices in accordance with aspects of thisdisclosure, process 180 is described herein with respect to videodecoder 30 of FIGS. 1 and 3, for the purpose of ease of description.Process 180 may begin when video decoder 30 determines a number ofentries included in a palette used to represent pixel values of a blockof video data to be decoded (182). Additionally, video decoder 30 maydetermine whether the block of video data includes at least one escapepixel that is not associated with any of the entries of the palette(184). For instance, if the color information of a pixel of the blockdoes not map to any entry of the palette, video decoder 30 may identifysuch a pixel as an escape pixel. In various examples, video decoder 30may identify the escape pixel using a flag signaled by video encoder 20or by an index value (e.g., “other index” described above) signaled byvideo encoder 20.

In turn, video decoder 30 may, responsive to determining that the numberof entries included in the palette is equal to one and that the block ofvideo data does not include at least one escape pixel, bypass decodingof index values associated with the palette for the pixel values of theblock of video data (186). As one example, video decoder 30 may receive,as part of an encoded video bitstream, encoded video data, e.g., syntaxelements and/or flags, associated with the block of video data, whereinthe encoded video data associated with the block does not include indexvalues associated with the palette for the pixel values of the block.Additionally, video decoder 30 may determine the pixel values of theblock of video data to be equal to the one entry included in the palette(188). For instance, video decoder 30 may reconstruct the block byassigning all pixels of the block the color information indicated by thesingle entry of the palette.

In one example, video decoder 30 may further receive, as part of anencoded video bitstream, a flag that indicates whether the index valuesare encoded for the block video data. In one example, to determine thenumber of entries included in the palette, video decoder 30 may receive,as part of an encoded video bitstream, a flag that indicates whether thenumber of entries in the palette is equal to one. In one example, todetermine whether the block of video data includes at least one escapepixel, video decoder 30 may receive, as part of an encoded videobitstream, a flag that indicates whether the block of video dataincludes at least one escape pixel. In one example, video decoder 30 mayreceive, as part of an encoded video bitstream, one or more syntaxelements associated with the palette. In this example, video decoder 30may decode the one or more syntax elements associated with the palette.

FIG. 5 is a flowchart illustrating an example process 200 by which avideo encoding device may implement techniques of this disclosure tobypass encoding of index values for pixels of a palette-coded block,based on a particular set of conditions. While process 200 may beperformed by a variety of devices in accordance with aspects of thisdisclosure, process 200 is described herein with respect to videoencoder 20 of FIGS. 1 and 2, for the purpose of ease of description.Process 200 may begin when video encoder 20 determines a number ofentries included in a palette used to represent pixel values of a blockof video data to be encoded (202). Additionally, video encoder 20 maydetermine whether the block of video data includes at least one escapepixel that is not associated with any of the entries in palette (204).

In turn, video encoder 20 may, responsive to determining that the numberof entries included in the palette is equal to one and that the block ofvideo data does not include at least one escape pixel, determine thatthe pixel values of the block are equal to the one entry of the palette,and bypass encoding of index values associated with the palette for thepixel values of the block of video data (206). For instance, videoencoding device 20 may encode data, e.g., syntax elements and/or flags,for the block without encoding index values mapping pixel values of theblock to entries in the palette corresponding to the block.Additionally, video encoding device 20 may encode one or more syntaxelements associated with the block of video data (208).

In one example, to encode the one or more syntax elements, video encoder20 may encode, as part of an encoded video bitstream, a flag thatindicates whether the index values are encoded for the pixel values ofthe block of video data. In one example, to encode the one or moresyntax elements, video encoder 20 may encode, as part of an encodedvideo bitstream, a flag that indicates whether the size of the paletteis equal to one. In one example, to encode the one or more syntaxelements, video encoder 20 may encode, as part of an encoded videobitstream, a flag that indicates whether the block of video dataincludes at least one escape pixel. In one example, video encoder 20 mayencode one or more syntax elements associated with the palette. In thisexample, video encoder 20 may signal, as part of an encoded videobitstream, the one or more syntax elements associated with the palette.

FIG. 6 is a flowchart illustrating an example process 220 by which avideo decoding device may implement techniques of this disclosure todequantize one or more escape pixels of a palette-coded block of videodata. While process 220 may be performed by a variety of devices inaccordance with aspects of this disclosure, process 220 is describedherein with respect to video decoder 30 of FIGS. 1 and 3, for thepurpose of ease of description. Process 220 may begin when video decoder30 determines a palette used to represent pixel values of a block ofvideo data to be decoded (222). Additionally, video decoder 30 mayidentify, in the block of video data, one or more escape pixels that arenot associated with any of one or more entries in the palette (224).

In turn, video decoder 30 may identify a single quantization parameter(QP) value for all of the one or more escape pixels of the block for agiven color channel based on a QP value used for transform coefficientcoding in non-palette based coding (226). For instance, video decoder 30may determine that the single QP value is equal to a QP value used fortraditional coefficient decoding of a color channel associated with theblock. Additionally, video decoder 30 may dequantize each of the one ormore escape pixels using the identified single QP value (228). In turn,video decoder 30 may determine the pixel values for the block of videodata based on the dequantized escape pixels and index values receivedfor one or more pixels in the block of video data that are associatedwith the one or more entries in the palette (230).

In one example, any two entries of the palette vary by at least apalette error limit. In one example, the palette error limit is directlyproportional to a palette QP value associated with the block. In oneexample, to identify the one or more escape pixels, video decoder 30 mayreceive, in an encoded video bitstream, a one-bit flag associated witheach of the one or more escape pixels, and determine, based on a valueof each received one-bit flag, that each of the one or more escapepixels is not associated with any of the entries of the palette. In oneexample, to determine that each of the one or more escape pixels is notassociated with any of the entries of the palette, video decoder 30 maydetermine that each of the one or more escape pixels is not associatedwith any of the entries of the palette based on the value of eachreceived one-bit flag and without decoding a pre-defined other indexvalue associated with escape pixels.

FIG. 7 is a flowchart illustrating an example process 240 by which avideo encoding device may implement techniques of this disclosure toquantize one or more escape pixels of a palette-coded block of videodata. While process 240 may be performed by a variety of devices inaccordance with aspects of this disclosure, process 240 is describedherein with respect to video encoder 20 of FIGS. 1 and 2, for thepurpose of ease of description. Process 200 may begin when video encoder20 determines a palette used to represent pixel values of a block ofvideo data to be encoded (242). Additionally, video encoder 20 mayidentify, in the block of video data, one or more escape pixels that arenot associated with any of one or more entries in the palette (244).

In turn, video encoder 20 may identify a single quantization parameter(QP) value for all of the one or more escape pixels of the block (246).For instance, video encoder 20 may determine that the single QP value isequal to a QP value used for traditional coefficient encoding of a colorchannel associated with the block. Additionally, video encoder 20 mayquantize each of the one or more escape pixels using the identifiedsingle QP value (228).

In one example, to identify the single QP value, video encoder 20 maydetermine that the single QP value is equal to a QP value used fortraditional coefficient encoding of a color channel associated with theblock. In one example, video encoder 20 may determine that a paletteerror limit of the palette is directly proportional to a QP valueassociated with the block, where any two entries of the palette are varyby at least a palette error limit. In one example, to determine thepalette error limit, video encoder 20 may identify the palette errorlimit using a table that maps the palette error limit to the QP valueassociated with the block. In one example, video encoder 20 may encode aone-bit flag associated with each of the one or more escape pixelswithout encoding a pre-defined other index value associated with escapepixels, wherein a value of each one-bit flag indicates that a respectiveone of the one or more escape pixels is not associated with any of theentries of the palette. In one example, to quantize each of the one ormore escape pixels using the identified single QP value, video encoder20 may solve a function that is based on the identified single QP value.In one such example, to solve the function, video encoder 20 may performa right-shift operation that is based on the identified single QP value.

In some examples, the techniques for palette-based coding of video datamay be used with one or more other coding techniques, such as techniquesfor inter- or intra-predictive coding. For example, as described ingreater detail below, an encoder or decoder, or combined encoder-decoder(codec), may be configured to perform inter- and intra-predictivecoding, as well as palette-based coding.

It is to be recognized that depending on the example, certain acts orevents of any of the techniques described herein can be performed in adifferent sequence, may be added, merged, or left out altogether (e.g.,not all described acts or events are necessary for the practice of thetechniques). Moreover, in certain examples, acts or events may beperformed concurrently, e.g., through multi-threaded processing,interrupt processing, or multiple processors, rather than sequentially.In addition, while certain aspects of this disclosure are described asbeing performed by a single module or unit for purposes of clarity, itshould be understood that the techniques of this disclosure may beperformed by a combination of units or modules associated with a videocoder.

Certain aspects of this disclosure have been described with respect tothe developing HEVC standard for purposes of illustration. However, thetechniques described in this disclosure may be useful for other videocoding processes, including other standard or proprietary video codingprocesses not yet developed.

The techniques described above may be performed by video encoder 20(FIGS. 1 and 2) and/or video decoder 30 (FIGS. 1 and 3), both of whichmay be generally referred to as a video coder. Likewise, video codingmay refer to video encoding or video decoding, as applicable.

While particular combinations of various aspects of the techniques aredescribed above, these combinations are provided merely to illustrateexamples of the techniques described in this disclosure. Accordingly,the techniques of this disclosure should not be limited to these examplecombinations and may encompass any conceivable combination of thevarious aspects of the techniques described in this disclosure.

In one or more examples, the functions described may be implemented inhardware, software, firmware, or any combination thereof. If implementedin software, the functions may be stored on or transmitted over, as oneor more instructions or code, a computer-readable medium and executed bya hardware-based processing unit. Computer-readable media may includecomputer-readable storage media, which corresponds to a tangible mediumsuch as data storage media, or communication media including any mediumthat facilitates transfer of a computer program from one place toanother, e.g., according to a communication protocol. In this manner,computer-readable media generally may correspond to (1) tangiblecomputer-readable storage media which is non-transitory or (2) acommunication medium such as a signal or carrier wave. Data storagemedia may be any available media that can be accessed by one or morecomputers or one or more processors to retrieve instructions, codeand/or data structures for implementation of the techniques described inthis disclosure. A computer program product may include acomputer-readable medium.

By way of example, and not limitation, such computer-readable storagemedia can comprise RAM, ROM, EEPROM, CD-ROM or other optical diskstorage, magnetic disk storage, or other magnetic storage devices, flashmemory, or any other medium that can be used to store desired programcode in the form of instructions or data structures and that can beaccessed by a computer. Also, any connection is properly termed acomputer-readable medium. For example, if instructions are transmittedfrom a website, server, or other remote source using a coaxial cable,fiber optic cable, twisted pair, digital subscriber line (DSL), orwireless technologies such as infrared, radio, and microwave, then thecoaxial cable, fiber optic cable, twisted pair, DSL, or wirelesstechnologies such as infrared, radio, and microwave are included in thedefinition of medium. It should be understood, however, thatcomputer-readable storage media and data storage media do not includeconnections, carrier waves, signals, or other transient media, but areinstead directed to non-transient, tangible storage media. Disk anddisc, as used herein, includes compact disc (CD), laser disc, opticaldisc, digital versatile disc (DVD), floppy disk and Blu-ray disc, wheredisks usually reproduce data magnetically, while discs reproduce dataoptically with lasers. Combinations of the above should also be includedwithin the scope of computer-readable media.

Instructions may be executed by one or more processors, such as one ormore digital signal processors (DSPs), general purpose microprocessors,application specific integrated circuits (ASICs), field programmablegate arrays (FPGAs), or other equivalent integrated or discrete logiccircuitry. Accordingly, the term “processor,” as used herein may referto any of the foregoing structure or any other structure suitable forimplementation of the techniques described herein. In addition, in someaspects, the functionality described herein may be provided withindedicated hardware and/or software modules configured for encoding anddecoding, or incorporated in a combined codec. Also, the techniquescould be fully implemented in one or more circuits or logic elements.

The techniques of this disclosure may be implemented in a wide varietyof devices or apparatuses, including a wireless handset, an integratedcircuit (IC) or a set of ICs (e.g., a chip set). Various components,modules, or units are described in this disclosure to emphasizefunctional aspects of devices configured to perform the disclosedtechniques, but do not necessarily require realization by differenthardware units. Rather, as described above, various units may becombined in a codec hardware unit or provided by a collection ofinteroperative hardware units, including one or more processors asdescribed above, in conjunction with suitable software and/or firmware.

Various examples have been described. These and other examples arewithin the scope of the following claims.

What is claimed is:
 1. A method of decoding video data, the methodcomprising: determining a palette used to represent pixel values of ablock of video data to be decoded; identifying, in the block of videodata, one or more escape pixels that are not associated with any of oneor more entries in the palette; identifying a single quantizationparameter (QP) value for all of the one or more escape pixels of theblock for a given color channel based on a QP value used for transformcoefficient coding in non-palette based coding; dequantizing each of theone or more escape pixels using the identified single QP value; anddetermining the pixel values of the block of video data based on thedequantized escape pixels and index values received for one or morepixels in the block of video data that are associated with the one ormore entries in the palette.
 2. The method of claim 1, wherein any twoentries of the palette vary by at least a palette error limit, andwherein the palette error limit is directly proportional to a palette QPvalue associated with the block.
 3. The method of claim 1, whereinidentifying the one or more escape pixels comprises: receiving, in anencoded video bitstream, a one-bit flag associated with each of the oneor more escape pixels; and determining, based on a value of eachreceived one-bit flag, that each of the one or more escape pixels is notassociated with any of the entries of the palette.
 4. The method ofclaim 3, wherein determining that each of the one or more escape pixelsis not associated with any of the entries of the palette comprisesdetermining that each of the one or more escape pixels is not associatedwith any of the entries of the palette based on the value of eachreceived one-bit flag and without decoding a pre-defined other indexvalue associated with escape pixels.
 5. A method of encoding video data,the method comprising: determining a palette used to represent pixelvalues of a block of video data to be encoded; identifying, in the blockof video data, one or more escape pixels that are not associated withany of the one or more entries in the palette; identifying a singlequantization parameter (QP) value for all of the one or more escapepixels of the block for a given color channel based on a QP value usedfor transform coefficient coding in non-palette based coding; quantizingeach of the one or more escape pixels using the identified single QPvalue; and encoding the pixel values of the block of video dataincluding the quantized escape pixels and index values for one or morepixels in the block of video data that are associated with the one ormore entries in the palette.
 6. The method of claim 5, whereinidentifying the single QP value comprises: determining that the singleQP value is equal to a QP value used for traditional coefficientencoding of a color channel associated with the block.
 7. The method ofclaim 5, further comprising: determining that a palette error limit ofthe palette is directly proportional to a QP value associated with theblock, wherein any two entries of the palette are vary by at least apalette error limit.
 8. The method of claim 7, wherein determining thepalette error limit comprises: identifying the palette error limit usinga table that maps the palette error limit to the QP value associatedwith the block.
 9. The method of claim 5, further comprising: encoding aone-bit flag associated with each of the one or more escape pixelswithout encoding a pre-defined other index value associated with escapepixels, wherein a value of each one-bit flag indicates that a respectiveone of the one or more escape pixels is not associated with any of theentries of the palette.
 10. The method of claim 5, wherein quantizingeach of the one or more escape pixels using the identified single QPvalue comprises solving a function that is based on the identifiedsingle QP value.
 11. The method of claim 10, wherein solving thefunction comprises performing a right-shift operation that is based onthe identified single QP value.
 12. A device for coding video data, thedevice comprising: a memory configured to store video data; and one ormore processors in communication with the memory and configured to:determine a palette used to represent pixel values of a block of videodata to be coded; identify, in the block of video data, one or moreescape pixels that are not associated with any of one or more entries inthe palette; identify a single quantization parameter (QP) value for allof the one or more escape pixels of the block for a given color channelbased on a QP value used for transform coefficient coding in non-palettebased coding; apply the identified single QP value to each of the one ormore escape pixels; and determine the pixel values of the block of videodata based on the escape pixels and index values received for one ormore pixels in the block of video data that are associated with the oneor more entries.
 13. The device of claim 12, wherein the devicecomprises a video decoding device, and wherein, to apply the identifiedsingle QP value to each of the one or more escape pixels, the one ormore processors are configured to dequantize each of the one or moreescape pixels using the identified single QP value.
 14. The device ofclaim 12, wherein to apply the identified single QP value to each of theone or more escape pixels, the one or more processors are configured tosolve a function that comprises performance of a right-shift operationbased on the identified single QP value, wherein the function.
 15. Thedevice of claim 12, wherein any two entries of the palette are vary byat least a palette error limit, and wherein the palette error limit isdirectly proportional to a palette QP value associated with the block.16. The device of claim 15, wherein the device comprises a device forencoding video data, and wherein the one or more processors are furtherconfigured to: identify the palette error limit using a table that mapsthe palette error limit to the QP value associated with the block. 17.The device of claim 12, wherein the one or more processors are furtherconfigured to identify each of the one or more escape pixels using arespective one-bit flag.
 18. The device of claim 12 further comprising avideo decoding device, wherein the one or more processors are furtherconfigured to: receive, in an encoded video bitstream, the respectiveone-bit flag associated with each of the one or more escape pixels; anddetermine, based on a value of each received one-bit flag and withoutdecoding a pre-defined other index value associated with escape pixels,that each of the one or more escape pixels is not associated with any ofthe entries of the palette.
 19. A computer-readable storage mediumencoded with instructions that, when executed, cause one or moreprocessors of a computing device to: determine a palette used torepresent pixel values of a block of video data to be coded; identify,in the block of video data, one or more escape pixels that are notassociated with any of one or more entries in the palette; identify asingle quantization parameter (QP) value for all of the one or moreescape pixels of the block for a given color channel based on a QP valueused for transform coefficient coding in non-palette based coding; applythe identified single QP value to each of the one or more escape pixels;and determine the pixel values of the block of video data based on theescape pixels and index values received for one or more pixels in theblock of video data that are associated with the one or more entries.20. An apparatus comprising: means for determining a palette used torepresent pixel values of a block of video data to be coded; means foridentifying, in the block of video data, one or more escape pixels thatare not associated with any of one or more entries in the palette; meansfor identifying a single quantization parameter (QP) value for all ofthe one or more escape pixels of the block for a given color channelbased on a QP value used for transform coefficient coding in non-palettebased coding; means for applying the identified single QP value to eachof the one or more escape pixels; and means for determining the pixelvalues of the block of video data based on the escape pixels and indexvalues received for one or more pixels in the block of video data thatare associated with the one or more entries.